2023-03-24T23:33:51.945 INFO:root:teuthology version: 0.0.1.dev1+ge372916 2023-03-24T23:33:51.946 DEBUG:teuthology.run:Teuthology command: teuthology --name pdonnell-2023-03-24_23:11:30-fs-wip-pdonnell-testing-20230323.162417-distro-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/data-scan} --archive /home/teuthworker/archive/pdonnell-2023-03-24_23:11:30-fs-wip-pdonnell-testing-20230323.162417-distro-default-smithi/7219632 --verbose --owner scheduled_pdonnell@teuthology -- /home/teuthworker/archive/pdonnell-2023-03-24_23:11:30-fs-wip-pdonnell-testing-20230323.162417-distro-default-smithi/7219632/orig.config.yaml 2023-03-24T23:33:51.992 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2023-03-24T23:33:52.067 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2023-03-24_23:11:30-fs-wip-pdonnell-testing-20230323.162417-distro-default-smithi/7219632 branch: wip-pdonnell-testing-20230323.162417 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/data-scan} email: pdonnell@redhat.com first_in_suite: false job_id: '7219632' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2023-03-24_23:11:30-fs-wip-pdonnell-testing-20230323.162417-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 8.stream overrides: admin_socket: branch: wip-pdonnell-testing-20230323.162417 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - bad backtrace - object missing on disk - error reading table object - error reading sessionmap - unmatched fragstat - unmatched rstat - was unreadable, recreating it now - Scrub error on inode - Metadata damage detected - inconsistent rstat on inode - Error recovering journal sha1: c19e8f2a497e95f75abc43ec14bc6095e30dd2cb ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: c19e8f2a497e95f75abc43ec14bc6095e30dd2cb selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20230323.162417 sha1: 1e7de2f05230c7419acf5ca7f5fd22964e30ab77 owner: scheduled_pdonnell@teuthology priority: 55 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 sha1: c19e8f2a497e95f75abc43ec14bc6095e30dd2cb sleep_before_teardown: 0 suite: fs suite_branch: wip-pdonnell-testing-20230323.162417 suite_path: /home/teuthworker/src/github.com_batrick_ceph_1e7de2f05230c7419acf5ca7f5fd22964e30ab77/qa suite_relpath: qa suite_repo: https://github.com/batrick/ceph.git suite_sha1: 1e7de2f05230c7419acf5ca7f5fd22964e30ab77 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi072.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwaltmMfrsNs7H/qyiEC7tdKlCS2yuw0lbX5+T5Gw7rzxvOkFR7S+V/P43cwZ3fPw+pzl6kG3TstpXwZ3YRQBi0mDatyWptWVfOgqQfvj6C1kGezTtZ678cV4ZKOmzq+dH1Idi87dKTpmT8AF0Ghm2poDrj3ommn6jN2ldA/30808GTS2s6O67cne9qP513dQzBKvVmxH1Da37nkwgddYnP7BlaWgNzhZNM95l2zkVnhxlRYjyNrWxdpXCCOPs+BafbAQQNaM0SG5CVCHgn01ehouk1J5CZVbO8QosFGW9plxdz+npUxxU6YZD/PrSEEhjURwv2y8SYlP+kN/t09eJvfltdPA98g+IMmq0MpEyURirDRiZyjKcBOwb//I9Rm2B5ntWJPj0bRatPyIiteMNwMHO0kA1+UIjVAotUrvKFn/S67brBe3RCE42OagZNjAan6/9ZUWcHcKFKtFWHew0ie6+xMHvyzejw44WZihJeWIwH9FI/psL6+E9YaYPIWc= smithi146.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMkAiVyNxV/ORARn9WKQ1l5a9Vj0+9RCo5UfCigB6duoMmd7Bn90XfZOYN9e7x2VGlH+qHDjXfEJHtLLeNSKWdQ= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_data_scan teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: parallel-gzip teuthology_sha1: fe9b6e52013911547168fef4a8506325b6c76139 timestamp: 2023-03-24_23:11:30 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.4538 2023-03-24T23:33:52.068 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_batrick_ceph_1e7de2f05230c7419acf5ca7f5fd22964e30ab77/qa; will attempt to use it 2023-03-24T23:33:52.069 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_batrick_ceph_1e7de2f05230c7419acf5ca7f5fd22964e30ab77/qa/tasks 2023-03-24T23:33:52.069 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-03-24T23:33:52.071 INFO:teuthology.task.internal:Checking packages... 2023-03-24T23:33:52.100 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'c19e8f2a497e95f75abc43ec14bc6095e30dd2cb' 2023-03-24T23:33:52.100 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-03-24T23:33:52.101 INFO:teuthology.packaging:ref: None 2023-03-24T23:33:52.101 INFO:teuthology.packaging:tag: None 2023-03-24T23:33:52.101 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20230323.162417 2023-03-24T23:33:52.101 INFO:teuthology.packaging:sha1: c19e8f2a497e95f75abc43ec14bc6095e30dd2cb 2023-03-24T23:33:52.102 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-pdonnell-testing-20230323.162417 2023-03-24T23:33:52.264 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-3124.gc19e8f2a 2023-03-24T23:33:52.264 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-03-24T23:33:52.270 INFO:teuthology.task.internal:no buildpackages task found 2023-03-24T23:33:52.270 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-03-24T23:33:52.280 INFO:teuthology.task.internal:Saving configuration 2023-03-24T23:33:52.296 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-03-24T23:33:52.302 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-03-24T23:33:52.328 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi072.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-03-24_23:11:30-fs-wip-pdonnell-testing-20230323.162417-distro-default-smithi/7219632', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-03-24 23:28:11.627086', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPzInAJE2p/7w8iUH+CGd6XOLFhTJfXiZZfMB99Uk/kP68xFh5x1SjREbqsUZWS8sbZ1ICHRf3s7ZRjiWveCHJY='} 2023-03-24T23:33:52.349 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi146.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-03-24_23:11:30-fs-wip-pdonnell-testing-20230323.162417-distro-default-smithi/7219632', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-03-24 23:28:11.628696', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMkAiVyNxV/ORARn9WKQ1l5a9Vj0+9RCo5UfCigB6duoMmd7Bn90XfZOYN9e7x2VGlH+qHDjXfEJHtLLeNSKWdQ='} 2023-03-24T23:33:52.350 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-03-24T23:33:52.356 INFO:teuthology.task.internal:roles: ubuntu@smithi072.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-03-24T23:33:52.356 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-03-24T23:33:52.356 INFO:teuthology.run_tasks:Running task console_log... 2023-03-24T23:33:52.428 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f55451e45e0>, signals=[15]) 2023-03-24T23:33:52.430 INFO:teuthology.run_tasks:Running task internal.connect... 2023-03-24T23:33:52.436 INFO:teuthology.task.internal:Opening connections... 2023-03-24T23:33:52.436 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi072.front.sepia.ceph.com 2023-03-24T23:33:52.438 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-24T23:33:52.506 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi146.front.sepia.ceph.com 2023-03-24T23:33:52.507 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-24T23:33:52.577 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-03-24T23:33:52.583 DEBUG:teuthology.orchestra.run.smithi072:> uname -m 2023-03-24T23:33:52.604 INFO:teuthology.orchestra.run.smithi072.stdout:x86_64 2023-03-24T23:33:52.604 DEBUG:teuthology.orchestra.run.smithi072:> cat /etc/os-release 2023-03-24T23:33:52.662 INFO:teuthology.orchestra.run.smithi072.stdout:NAME="CentOS Stream" 2023-03-24T23:33:52.662 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION="8" 2023-03-24T23:33:52.662 INFO:teuthology.orchestra.run.smithi072.stdout:ID="centos" 2023-03-24T23:33:52.662 INFO:teuthology.orchestra.run.smithi072.stdout:ID_LIKE="rhel fedora" 2023-03-24T23:33:52.662 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION_ID="8" 2023-03-24T23:33:52.663 INFO:teuthology.orchestra.run.smithi072.stdout:PLATFORM_ID="platform:el8" 2023-03-24T23:33:52.663 INFO:teuthology.orchestra.run.smithi072.stdout:PRETTY_NAME="CentOS Stream 8" 2023-03-24T23:33:52.663 INFO:teuthology.orchestra.run.smithi072.stdout:ANSI_COLOR="0;31" 2023-03-24T23:33:52.663 INFO:teuthology.orchestra.run.smithi072.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-03-24T23:33:52.663 INFO:teuthology.orchestra.run.smithi072.stdout:HOME_URL="https://centos.org/" 2023-03-24T23:33:52.663 INFO:teuthology.orchestra.run.smithi072.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-03-24T23:33:52.663 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-03-24T23:33:52.663 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-03-24T23:33:52.664 INFO:teuthology.lock.ops:Updating smithi072.front.sepia.ceph.com on lock server 2023-03-24T23:33:52.693 DEBUG:teuthology.orchestra.run.smithi146:> uname -m 2023-03-24T23:33:52.714 INFO:teuthology.orchestra.run.smithi146.stdout:x86_64 2023-03-24T23:33:52.714 DEBUG:teuthology.orchestra.run.smithi146:> cat /etc/os-release 2023-03-24T23:33:52.774 INFO:teuthology.orchestra.run.smithi146.stdout:NAME="CentOS Stream" 2023-03-24T23:33:52.774 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION="8" 2023-03-24T23:33:52.775 INFO:teuthology.orchestra.run.smithi146.stdout:ID="centos" 2023-03-24T23:33:52.775 INFO:teuthology.orchestra.run.smithi146.stdout:ID_LIKE="rhel fedora" 2023-03-24T23:33:52.775 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_ID="8" 2023-03-24T23:33:52.775 INFO:teuthology.orchestra.run.smithi146.stdout:PLATFORM_ID="platform:el8" 2023-03-24T23:33:52.775 INFO:teuthology.orchestra.run.smithi146.stdout:PRETTY_NAME="CentOS Stream 8" 2023-03-24T23:33:52.775 INFO:teuthology.orchestra.run.smithi146.stdout:ANSI_COLOR="0;31" 2023-03-24T23:33:52.775 INFO:teuthology.orchestra.run.smithi146.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-03-24T23:33:52.775 INFO:teuthology.orchestra.run.smithi146.stdout:HOME_URL="https://centos.org/" 2023-03-24T23:33:52.776 INFO:teuthology.orchestra.run.smithi146.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-03-24T23:33:52.776 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-03-24T23:33:52.776 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-03-24T23:33:52.776 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server 2023-03-24T23:33:52.801 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-03-24T23:33:52.817 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-03-24T23:33:52.822 INFO:teuthology.task.internal:Checking for old test directory... 2023-03-24T23:33:52.823 DEBUG:teuthology.orchestra.run.smithi072:> test '!' -e /home/ubuntu/cephtest 2023-03-24T23:33:52.825 DEBUG:teuthology.orchestra.run.smithi146:> test '!' -e /home/ubuntu/cephtest 2023-03-24T23:33:52.843 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-03-24T23:33:52.849 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-03-24T23:33:52.849 DEBUG:teuthology.orchestra.run.smithi072:> test -z $(ls -A /var/lib/ceph) 2023-03-24T23:33:52.882 DEBUG:teuthology.orchestra.run.smithi146:> test -z $(ls -A /var/lib/ceph) 2023-03-24T23:33:52.906 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-03-24T23:33:52.969 INFO:teuthology.run_tasks:Running task kernel... 2023-03-24T23:33:52.985 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-03-24T23:33:52.986 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-03-24T23:33:52.986 DEBUG:teuthology.orchestra.run.smithi072:> test -f /run/.containerenv -o -f /.dockerenv 2023-03-24T23:33:52.987 DEBUG:teuthology.orchestra.run.smithi146:> test -f /run/.containerenv -o -f /.dockerenv 2023-03-24T23:33:53.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-24T23:33:53.004 DEBUG:teuthology.orchestra.run.smithi072:> uname -r 2023-03-24T23:33:53.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-24T23:33:53.006 DEBUG:teuthology.orchestra.run.smithi146:> uname -r 2023-03-24T23:33:53.062 INFO:teuthology.orchestra.run.smithi072.stdout:4.18.0-408.el8.x86_64 2023-03-24T23:33:53.063 INFO:teuthology.task.kernel:Running kernel on smithi072: 4.18.0-408.el8.x86_64 2023-03-24T23:33:53.063 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum install -y kernel 2023-03-24T23:33:53.064 INFO:teuthology.orchestra.run.smithi146.stdout:4.18.0-408.el8.x86_64 2023-03-24T23:33:53.064 INFO:teuthology.task.kernel:Running kernel on smithi146: 4.18.0-408.el8.x86_64 2023-03-24T23:33:53.064 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum install -y kernel 2023-03-24T23:33:56.652 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS-8 - AppStream 38 MB/s | 29 MB 00:00 2023-03-24T23:33:59.074 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS-8 - AppStream 32 MB/s | 29 MB 00:00 2023-03-24T23:34:02.426 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS-8 - BaseOS 42 MB/s | 28 MB 00:00 2023-03-24T23:34:06.380 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS-8 - BaseOS 13 MB/s | 28 MB 00:02 2023-03-24T23:34:06.817 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS-8 - Extras 145 kB/s | 18 kB 00:00 2023-03-24T23:34:07.211 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS-8 - PowerTools 22 MB/s | 5.8 MB 00:00 2023-03-24T23:34:08.460 INFO:teuthology.orchestra.run.smithi146.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2023-03-24T23:34:09.502 INFO:teuthology.orchestra.run.smithi146.stdout:Extra Packages for Enterprise Linux 15 MB/s | 14 MB 00:00 2023-03-24T23:34:10.766 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS-8 - Extras 142 kB/s | 18 kB 00:00 2023-03-24T23:34:11.228 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS-8 - PowerTools 17 MB/s | 5.8 MB 00:00 2023-03-24T23:34:12.479 INFO:teuthology.orchestra.run.smithi072.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2023-03-24T23:34:13.211 INFO:teuthology.orchestra.run.smithi146.stdout:lab-extras 81 kB/s | 24 kB 00:00 2023-03-24T23:34:13.644 INFO:teuthology.orchestra.run.smithi072.stdout:Extra Packages for Enterprise Linux 13 MB/s | 14 MB 00:01 2023-03-24T23:34:16.525 INFO:teuthology.orchestra.run.smithi146.stdout:Package kernel-4.18.0-408.el8.x86_64 is already installed. 2023-03-24T23:34:16.526 INFO:teuthology.orchestra.run.smithi146.stdout:Package kernel-4.18.0-305.3.1.el8.x86_64 is already installed. 2023-03-24T23:34:16.698 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2023-03-24T23:34:16.699 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2023-03-24T23:34:16.699 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repository Size 2023-03-24T23:34:16.699 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2023-03-24T23:34:16.700 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2023-03-24T23:34:16.700 INFO:teuthology.orchestra.run.smithi146.stdout: kernel x86_64 4.18.0-448.el8 CentOS-BaseOS 9.2 M 2023-03-24T23:34:16.700 INFO:teuthology.orchestra.run.smithi146.stdout: kernel-core x86_64 4.18.0-448.el8 CentOS-BaseOS 41 M 2023-03-24T23:34:16.700 INFO:teuthology.orchestra.run.smithi146.stdout: kernel-modules x86_64 4.18.0-448.el8 CentOS-BaseOS 34 M 2023-03-24T23:34:16.700 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:34:16.700 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2023-03-24T23:34:16.700 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2023-03-24T23:34:16.701 INFO:teuthology.orchestra.run.smithi146.stdout:Install 3 Packages 2023-03-24T23:34:16.701 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:34:16.701 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 85 M 2023-03-24T23:34:16.702 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 94 M 2023-03-24T23:34:16.702 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2023-03-24T23:34:17.023 INFO:teuthology.orchestra.run.smithi072.stdout:lab-extras 481 kB/s | 24 kB 00:00 2023-03-24T23:34:17.062 INFO:teuthology.orchestra.run.smithi146.stdout:(1/3): kernel-4.18.0-448.el8.x86_64.rpm 25 MB/s | 9.2 MB 00:00 2023-03-24T23:34:17.870 INFO:teuthology.orchestra.run.smithi146.stdout:(2/3): kernel-core-4.18.0-448.el8.x86_64.rpm 35 MB/s | 41 MB 00:01 2023-03-24T23:34:18.270 INFO:teuthology.orchestra.run.smithi146.stdout:(3/3): kernel-modules-4.18.0-448.el8.x86_64.rpm 22 MB/s | 34 MB 00:01 2023-03-24T23:34:18.273 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2023-03-24T23:34:18.274 INFO:teuthology.orchestra.run.smithi146.stdout:Total 54 MB/s | 85 MB 00:01 2023-03-24T23:34:18.911 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2023-03-24T23:34:19.313 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2023-03-24T23:34:19.313 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2023-03-24T23:34:20.315 INFO:teuthology.orchestra.run.smithi072.stdout:Package kernel-4.18.0-408.el8.x86_64 is already installed. 2023-03-24T23:34:20.315 INFO:teuthology.orchestra.run.smithi072.stdout:Package kernel-4.18.0-305.3.1.el8.x86_64 is already installed. 2023-03-24T23:34:20.554 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-03-24T23:34:20.555 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-24T23:34:20.555 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2023-03-24T23:34:20.555 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-24T23:34:20.555 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-03-24T23:34:20.555 INFO:teuthology.orchestra.run.smithi072.stdout: kernel x86_64 4.18.0-448.el8 CentOS-BaseOS 9.2 M 2023-03-24T23:34:20.555 INFO:teuthology.orchestra.run.smithi072.stdout: kernel-core x86_64 4.18.0-448.el8 CentOS-BaseOS 41 M 2023-03-24T23:34:20.556 INFO:teuthology.orchestra.run.smithi072.stdout: kernel-modules x86_64 4.18.0-448.el8 CentOS-BaseOS 34 M 2023-03-24T23:34:20.556 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:34:20.556 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-03-24T23:34:20.556 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-24T23:34:20.556 INFO:teuthology.orchestra.run.smithi072.stdout:Install 3 Packages 2023-03-24T23:34:20.556 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:34:20.557 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 85 M 2023-03-24T23:34:20.557 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 94 M 2023-03-24T23:34:20.557 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-03-24T23:34:20.981 INFO:teuthology.orchestra.run.smithi072.stdout:(1/3): kernel-4.18.0-448.el8.x86_64.rpm 22 MB/s | 9.2 MB 00:00 2023-03-24T23:34:21.422 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2023-03-24T23:34:21.450 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2023-03-24T23:34:21.920 INFO:teuthology.orchestra.run.smithi072.stdout:(2/3): kernel-modules-4.18.0-448.el8.x86_64.rpm 25 MB/s | 34 MB 00:01 2023-03-24T23:34:22.836 INFO:teuthology.orchestra.run.smithi072.stdout:(3/3): kernel-core-4.18.0-448.el8.x86_64.rpm 18 MB/s | 41 MB 00:02 2023-03-24T23:34:22.839 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-03-24T23:34:22.839 INFO:teuthology.orchestra.run.smithi072.stdout:Total 37 MB/s | 85 MB 00:02 2023-03-24T23:34:23.474 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-03-24T23:34:23.980 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-03-24T23:34:23.980 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-03-24T23:34:24.164 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2023-03-24T23:34:24.824 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : kernel-core-4.18.0-448.el8.x86_64 1/3 2023-03-24T23:34:26.332 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-03-24T23:34:26.358 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-03-24T23:34:28.636 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 1/3 2023-03-24T23:34:29.092 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-03-24T23:34:29.098 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-03-24T23:34:29.838 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : kernel-core-4.18.0-448.el8.x86_64 1/3 2023-03-24T23:34:32.422 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-03-24T23:34:32.674 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : kernel-4.18.0-448.el8.x86_64 3/3 2023-03-24T23:34:33.651 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 1/3 2023-03-24T23:34:34.270 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-03-24T23:34:37.587 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-03-24T23:34:37.922 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : kernel-4.18.0-448.el8.x86_64 3/3 2023-03-24T23:34:54.319 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 3/3 2023-03-24T23:34:54.345 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-03-24T23:34:54.756 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: kernel-4.18.0-448.el8.x86_64 3/3 2023-03-24T23:34:54.756 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : kernel-4.18.0-448.el8.x86_64 1/3 2023-03-24T23:34:54.756 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : kernel-core-4.18.0-448.el8.x86_64 2/3 2023-03-24T23:34:55.357 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-03-24T23:34:55.357 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:34:55.357 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2023-03-24T23:34:55.357 INFO:teuthology.orchestra.run.smithi146.stdout: kernel-4.18.0-448.el8.x86_64 kernel-core-4.18.0-448.el8.x86_64 2023-03-24T23:34:55.357 INFO:teuthology.orchestra.run.smithi146.stdout: kernel-modules-4.18.0-448.el8.x86_64 2023-03-24T23:34:55.358 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:34:55.358 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2023-03-24T23:34:55.465 DEBUG:teuthology.orchestra.run.smithi146:> rpm -q kernel --last | head -n 1 2023-03-24T23:34:55.699 INFO:teuthology.orchestra.run.smithi146.stdout:kernel-4.18.0-448.el8.x86_64 Fri 24 Mar 2023 11:34:32 PM UTC 2023-03-24T23:34:55.700 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-03-24T23:34:55.701 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-408.el8.x86_64 Expected: 4.18.0-448.el8.x86_64 2023-03-24T23:34:55.701 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-03-24T23:34:55.701 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-03-24T23:34:55.701 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-03-24T23:34:55.701 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi146.front.sepia.ceph.com, path=None, version=distro) 2023-03-24T23:34:55.702 DEBUG:teuthology.orchestra.run.smithi146:> rpm -q kernel --last | head -n 1 2023-03-24T23:34:55.889 INFO:teuthology.orchestra.run.smithi146.stdout:kernel-4.18.0-448.el8.x86_64 Fri 24 Mar 2023 11:34:32 PM UTC 2023-03-24T23:34:55.891 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-03-24T23:34:55.891 DEBUG:teuthology.orchestra.run.smithi146:> sudo rpm -qi grub2-tools 2023-03-24T23:34:55.924 INFO:teuthology.orchestra.run.smithi146.stdout:Name : grub2-tools 2023-03-24T23:34:55.924 INFO:teuthology.orchestra.run.smithi146.stdout:Epoch : 1 2023-03-24T23:34:55.924 INFO:teuthology.orchestra.run.smithi146.stdout:Version : 2.02 2023-03-24T23:34:55.924 INFO:teuthology.orchestra.run.smithi146.stdout:Release : 129.el8 2023-03-24T23:34:55.925 INFO:teuthology.orchestra.run.smithi146.stdout:Architecture: x86_64 2023-03-24T23:34:55.925 INFO:teuthology.orchestra.run.smithi146.stdout:Install Date: Sun 28 Aug 2022 01:03:50 PM UTC 2023-03-24T23:34:55.925 INFO:teuthology.orchestra.run.smithi146.stdout:Group : System Environment/Base 2023-03-24T23:34:55.925 INFO:teuthology.orchestra.run.smithi146.stdout:Size : 9265586 2023-03-24T23:34:55.925 INFO:teuthology.orchestra.run.smithi146.stdout:License : GPLv3+ 2023-03-24T23:34:55.925 INFO:teuthology.orchestra.run.smithi146.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-03-24T23:34:55.925 INFO:teuthology.orchestra.run.smithi146.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-03-24T23:34:55.926 INFO:teuthology.orchestra.run.smithi146.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-03-24T23:34:55.926 INFO:teuthology.orchestra.run.smithi146.stdout:Build Host : kbuilder.bsys.centos.org 2023-03-24T23:34:55.926 INFO:teuthology.orchestra.run.smithi146.stdout:Relocations : (not relocatable) 2023-03-24T23:34:55.926 INFO:teuthology.orchestra.run.smithi146.stdout:Packager : CentOS BuildSystem 2023-03-24T23:34:55.926 INFO:teuthology.orchestra.run.smithi146.stdout:Vendor : CentOS 2023-03-24T23:34:55.926 INFO:teuthology.orchestra.run.smithi146.stdout:URL : http://www.gnu.org/software/grub/ 2023-03-24T23:34:55.926 INFO:teuthology.orchestra.run.smithi146.stdout:Summary : Support tools for GRUB. 2023-03-24T23:34:55.927 INFO:teuthology.orchestra.run.smithi146.stdout:Description : 2023-03-24T23:34:55.927 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:34:55.927 INFO:teuthology.orchestra.run.smithi146.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-03-24T23:34:55.927 INFO:teuthology.orchestra.run.smithi146.stdout:customizable bootloader with modular architecture. It supports a rich 2023-03-24T23:34:55.927 INFO:teuthology.orchestra.run.smithi146.stdout:variety of kernel formats, file systems, computer architectures and 2023-03-24T23:34:55.927 INFO:teuthology.orchestra.run.smithi146.stdout:hardware devices. 2023-03-24T23:34:55.927 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:34:55.927 INFO:teuthology.orchestra.run.smithi146.stdout:This subpackage provides tools for support of all platforms. 2023-03-24T23:34:55.928 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-03-24T23:34:55.929 INFO:teuthology.task.kernel:Updating grub on smithi146 to boot 4.18.0-448.el8.x86_64 2023-03-24T23:34:55.929 DEBUG:teuthology.orchestra.run.smithi146:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-03-24T23:34:57.576 INFO:teuthology.orchestra.run.smithi146.stderr:Generating grub configuration file ... 2023-03-24T23:34:59.342 INFO:teuthology.orchestra.run.smithi146.stderr:done 2023-03-24T23:34:59.344 DEBUG:teuthology.orchestra.run.smithi146:> mktemp 2023-03-24T23:34:59.399 INFO:teuthology.orchestra.run.smithi146.stdout:/tmp/tmp.KM6qE9VfGR 2023-03-24T23:34:59.400 DEBUG:teuthology.orchestra.run.smithi146:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.KM6qE9VfGR 2023-03-24T23:34:59.443 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 0666 /tmp/tmp.KM6qE9VfGR 2023-03-24T23:34:59.603 DEBUG:teuthology.orchestra.remote:smithi146:/tmp/tmp.KM6qE9VfGR is 6KB 2023-03-24T23:34:59.625 DEBUG:teuthology.orchestra.run.smithi146:> rm -fr /tmp/tmp.KM6qE9VfGR 2023-03-24T23:34:59.642 DEBUG:teuthology.orchestra.run.smithi146:> sudo /bin/ls /boot/loader/entries || true 2023-03-24T23:34:59.712 INFO:teuthology.orchestra.run.smithi146.stdout:2ff282740cf84d4fa02ff60ed95fbbff-0-rescue.conf 2023-03-24T23:34:59.712 INFO:teuthology.orchestra.run.smithi146.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-305.3.1.el8.x86_64.conf 2023-03-24T23:34:59.712 INFO:teuthology.orchestra.run.smithi146.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-408.el8.x86_64.conf 2023-03-24T23:34:59.712 INFO:teuthology.orchestra.run.smithi146.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64.conf 2023-03-24T23:34:59.713 DEBUG:teuthology.orchestra.run.smithi146:> sudo grub2-set-default 2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64 2023-03-24T23:34:59.889 DEBUG:teuthology.orchestra.run.smithi146:> sudo shutdown -r now 2023-03-24T23:35:00.751 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 3/3 2023-03-24T23:35:00.777 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-03-24T23:35:01.248 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: kernel-4.18.0-448.el8.x86_64 3/3 2023-03-24T23:35:01.249 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : kernel-4.18.0-448.el8.x86_64 1/3 2023-03-24T23:35:01.249 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : kernel-core-4.18.0-448.el8.x86_64 2/3 2023-03-24T23:35:01.848 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-03-24T23:35:01.849 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:35:01.849 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-03-24T23:35:01.849 INFO:teuthology.orchestra.run.smithi072.stdout: kernel-4.18.0-448.el8.x86_64 kernel-core-4.18.0-448.el8.x86_64 2023-03-24T23:35:01.849 INFO:teuthology.orchestra.run.smithi072.stdout: kernel-modules-4.18.0-448.el8.x86_64 2023-03-24T23:35:01.849 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:35:01.849 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-03-24T23:35:01.949 DEBUG:teuthology.orchestra.run.smithi072:> rpm -q kernel --last | head -n 1 2023-03-24T23:35:02.149 INFO:teuthology.orchestra.run.smithi072.stdout:kernel-4.18.0-448.el8.x86_64 Fri 24 Mar 2023 11:34:37 PM UTC 2023-03-24T23:35:02.151 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-03-24T23:35:02.151 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-408.el8.x86_64 Expected: 4.18.0-448.el8.x86_64 2023-03-24T23:35:02.152 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-03-24T23:35:02.152 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-03-24T23:35:02.152 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-03-24T23:35:02.152 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi072.front.sepia.ceph.com, path=None, version=distro) 2023-03-24T23:35:02.152 DEBUG:teuthology.orchestra.run.smithi072:> rpm -q kernel --last | head -n 1 2023-03-24T23:35:02.339 INFO:teuthology.orchestra.run.smithi072.stdout:kernel-4.18.0-448.el8.x86_64 Fri 24 Mar 2023 11:34:37 PM UTC 2023-03-24T23:35:02.340 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-03-24T23:35:02.341 DEBUG:teuthology.orchestra.run.smithi072:> sudo rpm -qi grub2-tools 2023-03-24T23:35:02.372 INFO:teuthology.orchestra.run.smithi072.stdout:Name : grub2-tools 2023-03-24T23:35:02.373 INFO:teuthology.orchestra.run.smithi072.stdout:Epoch : 1 2023-03-24T23:35:02.373 INFO:teuthology.orchestra.run.smithi072.stdout:Version : 2.02 2023-03-24T23:35:02.373 INFO:teuthology.orchestra.run.smithi072.stdout:Release : 129.el8 2023-03-24T23:35:02.374 INFO:teuthology.orchestra.run.smithi072.stdout:Architecture: x86_64 2023-03-24T23:35:02.374 INFO:teuthology.orchestra.run.smithi072.stdout:Install Date: Sun 28 Aug 2022 01:03:50 PM UTC 2023-03-24T23:35:02.374 INFO:teuthology.orchestra.run.smithi072.stdout:Group : System Environment/Base 2023-03-24T23:35:02.374 INFO:teuthology.orchestra.run.smithi072.stdout:Size : 9265586 2023-03-24T23:35:02.374 INFO:teuthology.orchestra.run.smithi072.stdout:License : GPLv3+ 2023-03-24T23:35:02.374 INFO:teuthology.orchestra.run.smithi072.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-03-24T23:35:02.374 INFO:teuthology.orchestra.run.smithi072.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-03-24T23:35:02.375 INFO:teuthology.orchestra.run.smithi072.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-03-24T23:35:02.375 INFO:teuthology.orchestra.run.smithi072.stdout:Build Host : kbuilder.bsys.centos.org 2023-03-24T23:35:02.375 INFO:teuthology.orchestra.run.smithi072.stdout:Relocations : (not relocatable) 2023-03-24T23:35:02.375 INFO:teuthology.orchestra.run.smithi072.stdout:Packager : CentOS BuildSystem 2023-03-24T23:35:02.375 INFO:teuthology.orchestra.run.smithi072.stdout:Vendor : CentOS 2023-03-24T23:35:02.375 INFO:teuthology.orchestra.run.smithi072.stdout:URL : http://www.gnu.org/software/grub/ 2023-03-24T23:35:02.375 INFO:teuthology.orchestra.run.smithi072.stdout:Summary : Support tools for GRUB. 2023-03-24T23:35:02.375 INFO:teuthology.orchestra.run.smithi072.stdout:Description : 2023-03-24T23:35:02.376 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:35:02.376 INFO:teuthology.orchestra.run.smithi072.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-03-24T23:35:02.376 INFO:teuthology.orchestra.run.smithi072.stdout:customizable bootloader with modular architecture. It supports a rich 2023-03-24T23:35:02.376 INFO:teuthology.orchestra.run.smithi072.stdout:variety of kernel formats, file systems, computer architectures and 2023-03-24T23:35:02.376 INFO:teuthology.orchestra.run.smithi072.stdout:hardware devices. 2023-03-24T23:35:02.376 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:35:02.376 INFO:teuthology.orchestra.run.smithi072.stdout:This subpackage provides tools for support of all platforms. 2023-03-24T23:35:02.377 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-03-24T23:35:02.378 INFO:teuthology.task.kernel:Updating grub on smithi072 to boot 4.18.0-448.el8.x86_64 2023-03-24T23:35:02.378 DEBUG:teuthology.orchestra.run.smithi072:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-03-24T23:35:04.440 INFO:teuthology.orchestra.run.smithi072.stderr:Generating grub configuration file ... 2023-03-24T23:35:07.180 INFO:teuthology.orchestra.run.smithi072.stderr:done 2023-03-24T23:35:07.182 DEBUG:teuthology.orchestra.run.smithi072:> mktemp 2023-03-24T23:35:07.218 INFO:teuthology.orchestra.run.smithi072.stdout:/tmp/tmp.4c4FCnmWgN 2023-03-24T23:35:07.218 DEBUG:teuthology.orchestra.run.smithi072:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.4c4FCnmWgN 2023-03-24T23:35:07.316 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 0666 /tmp/tmp.4c4FCnmWgN 2023-03-24T23:35:07.456 DEBUG:teuthology.orchestra.remote:smithi072:/tmp/tmp.4c4FCnmWgN is 6KB 2023-03-24T23:35:07.478 DEBUG:teuthology.orchestra.run.smithi072:> rm -fr /tmp/tmp.4c4FCnmWgN 2023-03-24T23:35:07.495 DEBUG:teuthology.orchestra.run.smithi072:> sudo /bin/ls /boot/loader/entries || true 2023-03-24T23:35:07.559 INFO:teuthology.orchestra.run.smithi072.stdout:2ff282740cf84d4fa02ff60ed95fbbff-0-rescue.conf 2023-03-24T23:35:07.559 INFO:teuthology.orchestra.run.smithi072.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-305.3.1.el8.x86_64.conf 2023-03-24T23:35:07.559 INFO:teuthology.orchestra.run.smithi072.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-408.el8.x86_64.conf 2023-03-24T23:35:07.559 INFO:teuthology.orchestra.run.smithi072.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64.conf 2023-03-24T23:35:07.560 DEBUG:teuthology.orchestra.run.smithi072:> sudo grub2-set-default 2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64 2023-03-24T23:35:07.718 DEBUG:teuthology.orchestra.run.smithi072:> sudo shutdown -r now 2023-03-24T23:35:29.909 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-03-24T23:35:29.910 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2023-03-24T23:35:29.910 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-24T23:35:37.725 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-03-24T23:35:37.725 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2023-03-24T23:35:37.726 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-24T23:35:48.229 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.146 2023-03-24T23:35:56.165 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.72 2023-03-24T23:36:18.249 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2023-03-24T23:36:18.250 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-24T23:36:21.317 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.146 2023-03-24T23:36:26.169 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2023-03-24T23:36:26.170 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-24T23:36:29.225 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.72 2023-03-24T23:36:51.337 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2023-03-24T23:36:51.338 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-24T23:36:52.533 DEBUG:teuthology.orchestra.run.smithi146:> true 2023-03-24T23:36:52.764 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi146.front.sepia.ceph.com' 2023-03-24T23:36:52.765 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "4.18.0-448.el8.x86_64"... 2023-03-24T23:36:52.765 DEBUG:teuthology.orchestra.run.smithi146:> uname -r 2023-03-24T23:36:52.822 INFO:teuthology.orchestra.run.smithi146.stdout:4.18.0-448.el8.x86_64 2023-03-24T23:36:52.823 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-448.el8.x86_64 vs 4.18.0-448.el8.x86_64 2023-03-24T23:36:52.823 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-03-24T23:36:52.823 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-03-24T23:36:53.824 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-03-24T23:36:53.825 DEBUG:teuthology.orchestra.run.smithi146:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-03-24T23:36:53.902 INFO:teuthology.orchestra.run.smithi146.stdout:ttyS1 2023-03-24T23:36:53.916 DEBUG:teuthology.parallel:result is None 2023-03-24T23:36:59.226 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2023-03-24T23:36:59.227 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-24T23:37:00.078 DEBUG:teuthology.orchestra.run.smithi072:> true 2023-03-24T23:37:01.042 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi072.front.sepia.ceph.com' 2023-03-24T23:37:01.042 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.18.0-448.el8.x86_64"... 2023-03-24T23:37:01.043 DEBUG:teuthology.orchestra.run.smithi072:> uname -r 2023-03-24T23:37:01.061 INFO:teuthology.orchestra.run.smithi072.stdout:4.18.0-448.el8.x86_64 2023-03-24T23:37:01.062 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-448.el8.x86_64 vs 4.18.0-448.el8.x86_64 2023-03-24T23:37:01.062 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-03-24T23:37:01.062 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-03-24T23:37:02.063 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-03-24T23:37:02.064 DEBUG:teuthology.orchestra.run.smithi072:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-03-24T23:37:02.170 INFO:teuthology.orchestra.run.smithi072.stdout:ttyS1 2023-03-24T23:37:02.182 DEBUG:teuthology.parallel:result is None 2023-03-24T23:37:02.182 INFO:teuthology.run_tasks:Running task internal.base... 2023-03-24T23:37:02.190 INFO:teuthology.task.internal:Creating test directory... 2023-03-24T23:37:02.190 DEBUG:teuthology.orchestra.run.smithi072:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-03-24T23:37:02.192 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-03-24T23:37:02.213 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-03-24T23:37:02.220 INFO:teuthology.run_tasks:Running task internal.archive... 2023-03-24T23:37:02.227 INFO:teuthology.task.internal:Creating archive directory... 2023-03-24T23:37:02.227 DEBUG:teuthology.orchestra.run.smithi072:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-03-24T23:37:02.250 DEBUG:teuthology.orchestra.run.smithi146:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-03-24T23:37:02.293 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-03-24T23:37:02.300 INFO:teuthology.task.internal:Enabling coredump saving... 2023-03-24T23:37:02.300 DEBUG:teuthology.orchestra.run.smithi072:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-03-24T23:37:02.321 DEBUG:teuthology.orchestra.run.smithi146:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-03-24T23:37:02.352 INFO:teuthology.orchestra.run.smithi072.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-03-24T23:37:02.361 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-03-24T23:37:02.362 INFO:teuthology.orchestra.run.smithi072.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-03-24T23:37:02.373 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-03-24T23:37:02.374 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-03-24T23:37:02.381 INFO:teuthology.task.internal:Configuring sudo... 2023-03-24T23:37:02.381 DEBUG:teuthology.orchestra.run.smithi072:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-03-24T23:37:02.405 DEBUG:teuthology.orchestra.run.smithi146:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-03-24T23:37:02.444 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-03-24T23:37:02.453 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-03-24T23:37:02.454 DEBUG:teuthology.orchestra.run.smithi072:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-03-24T23:37:02.474 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-03-24T23:37:02.502 DEBUG:teuthology.orchestra.run.smithi072:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-03-24T23:37:02.579 DEBUG:teuthology.orchestra.run.smithi072:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-03-24T23:37:02.649 DEBUG:teuthology.orchestra.run.smithi072:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-03-24T23:37:02.699 DEBUG:teuthology.orchestra.run.smithi072:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-03-24T23:37:02.735 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-24T23:37:02.736 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-03-24T23:37:02.802 DEBUG:teuthology.orchestra.run.smithi146:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-03-24T23:37:02.851 DEBUG:teuthology.orchestra.run.smithi146:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-03-24T23:37:02.901 DEBUG:teuthology.orchestra.run.smithi146:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-03-24T23:37:02.991 DEBUG:teuthology.orchestra.run.smithi146:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-03-24T23:37:03.030 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-03-24T23:37:03.030 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-03-24T23:37:03.097 DEBUG:teuthology.orchestra.run.smithi072:> sudo service rsyslog restart 2023-03-24T23:37:03.099 DEBUG:teuthology.orchestra.run.smithi146:> sudo service rsyslog restart 2023-03-24T23:37:03.151 INFO:teuthology.orchestra.run.smithi072.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-03-24T23:37:03.204 INFO:teuthology.orchestra.run.smithi146.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-03-24T23:37:03.670 INFO:teuthology.run_tasks:Running task internal.timer... 2023-03-24T23:37:03.677 INFO:teuthology.task.internal:Starting timer... 2023-03-24T23:37:03.677 INFO:teuthology.run_tasks:Running task pcp... 2023-03-24T23:37:03.688 INFO:teuthology.run_tasks:Running task selinux... 2023-03-24T23:37:03.698 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-03-24T23:37:03.699 DEBUG:teuthology.orchestra.run.smithi072:> sudo service auditd rotate 2023-03-24T23:37:03.785 INFO:teuthology.orchestra.run.smithi072.stdout:Rotating logs: 2023-03-24T23:37:03.788 DEBUG:teuthology.orchestra.run.smithi146:> sudo service auditd rotate 2023-03-24T23:37:03.840 INFO:teuthology.orchestra.run.smithi146.stdout:Rotating logs: 2023-03-24T23:37:03.842 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-03-24T23:37:03.842 DEBUG:teuthology.orchestra.run.smithi072:> /usr/sbin/getenforce 2023-03-24T23:37:03.869 INFO:teuthology.orchestra.run.smithi072.stdout:Permissive 2023-03-24T23:37:03.870 DEBUG:teuthology.orchestra.run.smithi146:> /usr/sbin/getenforce 2023-03-24T23:37:03.898 INFO:teuthology.orchestra.run.smithi146.stdout:Permissive 2023-03-24T23:37:03.898 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi072.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi146.front.sepia.ceph.com': 'permissive'} 2023-03-24T23:37:03.899 DEBUG:teuthology.orchestra.run.smithi072:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-03-24T23:37:03.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-24T23:37:03.940 DEBUG:teuthology.orchestra.run.smithi146:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-03-24T23:37:03.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-24T23:37:03.969 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-03-24T23:37:03.969 DEBUG:teuthology.orchestra.run.smithi072:> sudo /usr/sbin/setenforce permissive 2023-03-24T23:37:04.018 DEBUG:teuthology.orchestra.run.smithi146:> sudo /usr/sbin/setenforce permissive 2023-03-24T23:37:04.052 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-03-24T23:37:04.062 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-03-24T23:37:04.069 INFO:teuthology.repo_utils:Fetching main from origin 2023-03-24T23:37:04.227 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-03-24T23:37:04.246 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-03-24T23:37:04.247 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi072.front.sepia.ceph.com,smithi146.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-03-24T23:44:22.154 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi072.front.sepia.ceph.com'), Remote(name='ubuntu@smithi146.front.sepia.ceph.com')] 2023-03-24T23:44:22.156 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2023-03-24T23:44:22.157 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-24T23:44:22.228 DEBUG:teuthology.orchestra.run.smithi072:> true 2023-03-24T23:44:22.255 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2023-03-24T23:44:22.257 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-24T23:44:22.327 DEBUG:teuthology.orchestra.run.smithi146:> true 2023-03-24T23:44:22.355 INFO:teuthology.run_tasks:Running task clock... 2023-03-24T23:44:22.369 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-03-24T23:44:22.370 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-03-24T23:44:22.370 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-03-24T23:44:22.372 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-03-24T23:44:22.373 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-03-24T23:44:22.402 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-03-24T23:44:22.416 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-03-24T23:44:22.428 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-03-24T23:44:22.442 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-03-24T23:44:22.479 INFO:teuthology.orchestra.run.smithi072.stderr:sudo: ntpd: command not found 2023-03-24T23:44:22.490 INFO:teuthology.orchestra.run.smithi072.stdout:506 Cannot talk to daemon 2023-03-24T23:44:22.504 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-03-24T23:44:22.514 INFO:teuthology.orchestra.run.smithi146.stderr:sudo: ntpd: command not found 2023-03-24T23:44:22.518 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-03-24T23:44:22.526 INFO:teuthology.orchestra.run.smithi146.stdout:506 Cannot talk to daemon 2023-03-24T23:44:22.540 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-03-24T23:44:22.554 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-03-24T23:44:22.587 INFO:teuthology.orchestra.run.smithi072.stderr:bash: ntpq: command not found 2023-03-24T23:44:22.590 INFO:teuthology.orchestra.run.smithi072.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-03-24T23:44:22.591 INFO:teuthology.orchestra.run.smithi072.stdout:=============================================================================== 2023-03-24T23:44:22.591 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-24T23:44:22.591 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-24T23:44:22.591 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-24T23:44:22.591 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-24T23:44:22.618 INFO:teuthology.orchestra.run.smithi146.stderr:bash: ntpq: command not found 2023-03-24T23:44:22.622 INFO:teuthology.orchestra.run.smithi146.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-03-24T23:44:22.622 INFO:teuthology.orchestra.run.smithi146.stdout:=============================================================================== 2023-03-24T23:44:22.622 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-24T23:44:22.622 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-24T23:44:22.622 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-24T23:44:22.622 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-24T23:44:22.623 INFO:teuthology.run_tasks:Running task install... 2023-03-24T23:44:22.632 DEBUG:teuthology.task.install:project ceph 2023-03-24T23:44:22.633 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'c19e8f2a497e95f75abc43ec14bc6095e30dd2cb'}} 2023-03-24T23:44:22.633 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'c19e8f2a497e95f75abc43ec14bc6095e30dd2cb'} 2023-03-24T23:44:22.633 INFO:teuthology.task.install:Using flavor: default 2023-03-24T23:44:22.639 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-03-24T23:44:22.639 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-03-24T23:44:22.640 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c19e8f2a497e95f75abc43ec14bc6095e30dd2cb', 'tag': None, 'wait_for_package': False} 2023-03-24T23:44:22.641 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=c19e8f2a497e95f75abc43ec14bc6095e30dd2cb 2023-03-24T23:44:22.643 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c19e8f2a497e95f75abc43ec14bc6095e30dd2cb', 'tag': None, 'wait_for_package': False} 2023-03-24T23:44:22.643 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=c19e8f2a497e95f75abc43ec14bc6095e30dd2cb 2023-03-24T23:44:22.793 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230323.162417/c19e8f2a497e95f75abc43ec14bc6095e30dd2cb/centos/8/flavors/default/ 2023-03-24T23:44:22.793 INFO:teuthology.task.install.rpm:Package version is 18.0.0-3124.gc19e8f2a 2023-03-24T23:44:22.801 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230323.162417/c19e8f2a497e95f75abc43ec14bc6095e30dd2cb/centos/8/flavors/default/ 2023-03-24T23:44:22.801 INFO:teuthology.task.install.rpm:Package version is 18.0.0-3124.gc19e8f2a 2023-03-24T23:44:22.899 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230323.162417/c19e8f2a497e95f75abc43ec14bc6095e30dd2cb/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230323.162417/c19e8f2a497e95f75abc43ec14bc6095e30dd2cb/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230323.162417/c19e8f2a497e95f75abc43ec14bc6095e30dd2cb/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-03-24T23:44:22.899 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-24T23:44:22.900 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-03-24T23:44:22.903 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230323.162417/c19e8f2a497e95f75abc43ec14bc6095e30dd2cb/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230323.162417/c19e8f2a497e95f75abc43ec14bc6095e30dd2cb/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230323.162417/c19e8f2a497e95f75abc43ec14bc6095e30dd2cb/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-03-24T23:44:22.903 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-03-24T23:44:22.904 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-03-24T23:44:22.932 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-03-24T23:44:22.933 DEBUG:teuthology.orchestra.run.smithi072:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c19e8f2a497e95f75abc43ec14bc6095e30dd2cb/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-03-24T23:44:22.934 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-03-24T23:44:22.934 DEBUG:teuthology.orchestra.run.smithi146:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c19e8f2a497e95f75abc43ec14bc6095e30dd2cb/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-03-24T23:44:23.002 DEBUG:teuthology.orchestra.run.smithi146:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-03-24T23:44:23.003 DEBUG:teuthology.orchestra.run.smithi072:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-03-24T23:44:23.079 DEBUG:teuthology.orchestra.run.smithi146:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-03-24T23:44:23.081 DEBUG:teuthology.orchestra.run.smithi072:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-03-24T23:44:23.146 INFO:teuthology.orchestra.run.smithi146.stdout:check_obsoletes = 1 2023-03-24T23:44:23.147 INFO:teuthology.orchestra.run.smithi072.stdout:check_obsoletes = 1 2023-03-24T23:44:23.148 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum clean all 2023-03-24T23:44:23.149 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum clean all 2023-03-24T23:44:23.400 INFO:teuthology.orchestra.run.smithi146.stdout:71 files removed 2023-03-24T23:44:23.401 INFO:teuthology.orchestra.run.smithi072.stdout:71 files removed 2023-03-24T23:44:23.429 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-03-24T23:44:23.430 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-03-24T23:44:24.087 INFO:teuthology.orchestra.run.smithi072.stdout:ceph packages for x86_64 314 kB/s | 80 kB 00:00 2023-03-24T23:44:24.111 INFO:teuthology.orchestra.run.smithi146.stdout:ceph packages for x86_64 289 kB/s | 80 kB 00:00 2023-03-24T23:44:24.356 INFO:teuthology.orchestra.run.smithi072.stdout:ceph noarch packages 41 kB/s | 9.4 kB 00:00 2023-03-24T23:44:24.362 INFO:teuthology.orchestra.run.smithi146.stdout:ceph noarch packages 44 kB/s | 9.4 kB 00:00 2023-03-24T23:44:24.615 INFO:teuthology.orchestra.run.smithi072.stdout:ceph source packages 6.4 kB/s | 1.5 kB 00:00 2023-03-24T23:44:24.621 INFO:teuthology.orchestra.run.smithi146.stdout:ceph source packages 6.4 kB/s | 1.5 kB 00:00 2023-03-24T23:44:25.532 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS-8 - AppStream 32 MB/s | 29 MB 00:00 2023-03-24T23:44:25.671 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS-8 - AppStream 28 MB/s | 29 MB 00:01 2023-03-24T23:44:31.416 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS-8 - BaseOS 41 MB/s | 28 MB 00:00 2023-03-24T23:44:31.722 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS-8 - BaseOS 32 MB/s | 28 MB 00:00 2023-03-24T23:44:35.790 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS-8 - Extras 138 kB/s | 18 kB 00:00 2023-03-24T23:44:36.055 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS-8 - Extras 117 kB/s | 18 kB 00:00 2023-03-24T23:44:36.175 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS-8 - PowerTools 23 MB/s | 5.8 MB 00:00 2023-03-24T23:44:36.534 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS-8 - PowerTools 17 MB/s | 5.8 MB 00:00 2023-03-24T23:44:37.441 INFO:teuthology.orchestra.run.smithi146.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2023-03-24T23:44:37.785 INFO:teuthology.orchestra.run.smithi072.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2023-03-24T23:44:39.158 INFO:teuthology.orchestra.run.smithi146.stdout:Extra Packages for Enterprise Linux 8.7 MB/s | 14 MB 00:01 2023-03-24T23:44:39.670 INFO:teuthology.orchestra.run.smithi072.stdout:Extra Packages for Enterprise Linux 7.9 MB/s | 14 MB 00:01 2023-03-24T23:44:42.525 INFO:teuthology.orchestra.run.smithi146.stdout:lab-extras 342 kB/s | 24 kB 00:00 2023-03-24T23:44:43.029 INFO:teuthology.orchestra.run.smithi072.stdout:lab-extras 436 kB/s | 24 kB 00:00 2023-03-24T23:44:46.062 INFO:teuthology.orchestra.run.smithi146.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-03-24T23:44:46.063 INFO:teuthology.orchestra.run.smithi146.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-03-24T23:44:46.075 INFO:teuthology.orchestra.run.smithi146.stdout:Package NetworkManager-1:1.39.12-1.el8.x86_64 is already installed. 2023-03-24T23:44:46.075 INFO:teuthology.orchestra.run.smithi146.stdout:Package NetworkManager-initscripts-updown-1:1.39.12-1.el8.noarch is already installed. 2023-03-24T23:44:46.075 INFO:teuthology.orchestra.run.smithi146.stdout:Package iproute-5.18.0-1.el8.x86_64 is already installed. 2023-03-24T23:44:46.077 INFO:teuthology.orchestra.run.smithi146.stdout:Package util-linux-2.32.1-38.el8.x86_64 is already installed. 2023-03-24T23:44:46.077 INFO:teuthology.orchestra.run.smithi146.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-03-24T23:44:46.079 INFO:teuthology.orchestra.run.smithi146.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-03-24T23:44:46.082 INFO:teuthology.orchestra.run.smithi146.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-03-24T23:44:46.083 INFO:teuthology.orchestra.run.smithi146.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-03-24T23:44:46.216 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2023-03-24T23:44:46.229 INFO:teuthology.orchestra.run.smithi146.stdout:========================================================================================================= 2023-03-24T23:44:46.229 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repository Size 2023-03-24T23:44:46.230 INFO:teuthology.orchestra.run.smithi146.stdout:========================================================================================================= 2023-03-24T23:44:46.230 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2023-03-24T23:44:46.230 INFO:teuthology.orchestra.run.smithi146.stdout: bison x86_64 3.0.4-10.el8 CentOS-AppStream 688 k 2023-03-24T23:44:46.230 INFO:teuthology.orchestra.run.smithi146.stdout: ceph x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 6.4 k 2023-03-24T23:44:46.231 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-base x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 5.4 M 2023-03-24T23:44:46.231 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-fuse x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 846 k 2023-03-24T23:44:46.233 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 165 k 2023-03-24T23:44:46.233 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 1.5 M 2023-03-24T23:44:46.233 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-cephadm noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 123 k 2023-03-24T23:44:46.234 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-dashboard noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 1.7 M 2023-03-24T23:44:46.234 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 7.4 M 2023-03-24T23:44:46.234 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-rook noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 47 k 2023-03-24T23:44:46.234 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-radosgw x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 13 M 2023-03-24T23:44:46.234 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-test x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 41 M 2023-03-24T23:44:46.234 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-volume noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 263 k 2023-03-24T23:44:46.234 INFO:teuthology.orchestra.run.smithi146.stdout: cephadm noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 83 k 2023-03-24T23:44:46.235 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-mirror x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 258 k 2023-03-24T23:44:46.235 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-top noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 18 k 2023-03-24T23:44:46.235 INFO:teuthology.orchestra.run.smithi146.stdout: elfutils-libelf-devel x86_64 0.188-3.el8 CentOS-BaseOS 61 k 2023-03-24T23:44:46.235 INFO:teuthology.orchestra.run.smithi146.stdout: flex x86_64 2.6.1-9.el8 CentOS-AppStream 320 k 2023-03-24T23:44:46.235 INFO:teuthology.orchestra.run.smithi146.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 19 k 2023-03-24T23:44:46.235 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs-devel x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 30 k 2023-03-24T23:44:46.235 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs2 x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 769 k 2023-03-24T23:44:46.236 INFO:teuthology.orchestra.run.smithi146.stdout: librados-devel x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 139 k 2023-03-24T23:44:46.236 INFO:teuthology.orchestra.run.smithi146.stdout: libtool x86_64 2.4.6-25.el8 CentOS-AppStream 709 k 2023-03-24T23:44:46.236 INFO:teuthology.orchestra.run.smithi146.stdout: libuuid-devel x86_64 2.32.1-41.el8 CentOS-BaseOS 99 k 2023-03-24T23:44:46.236 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-devel x86_64 1:1.1.1k-9.el8 CentOS-BaseOS 3.2 M 2023-03-24T23:44:46.236 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql x86_64 10.23-1.module_el8+300+2c53004c CentOS-AppStream 1.6 M 2023-03-24T23:44:46.236 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-contrib x86_64 10.23-1.module_el8+300+2c53004c CentOS-AppStream 903 k 2023-03-24T23:44:46.236 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-server x86_64 10.23-1.module_el8+300+2c53004c CentOS-AppStream 5.1 M 2023-03-24T23:44:46.237 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cephfs x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 190 k 2023-03-24T23:44:46.237 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rados x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 370 k 2023-03-24T23:44:46.237 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rbd x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 364 k 2023-03-24T23:44:46.237 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rgw x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 107 k 2023-03-24T23:44:46.237 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-fuse x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 94 k 2023-03-24T23:44:46.237 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-mirror x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 3.4 M 2023-03-24T23:44:46.237 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-nbd x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 179 k 2023-03-24T23:44:46.238 INFO:teuthology.orchestra.run.smithi146.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 CentOS-BaseOS 72 k 2023-03-24T23:44:46.238 INFO:teuthology.orchestra.run.smithi146.stdout:Upgrading: 2023-03-24T23:44:46.238 INFO:teuthology.orchestra.run.smithi146.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-1.el8 CentOS-BaseOS 142 k 2023-03-24T23:44:46.238 INFO:teuthology.orchestra.run.smithi146.stdout: elfutils-debuginfod-client x86_64 0.188-3.el8 CentOS-BaseOS 75 k 2023-03-24T23:44:46.238 INFO:teuthology.orchestra.run.smithi146.stdout: elfutils-libelf x86_64 0.188-3.el8 CentOS-BaseOS 231 k 2023-03-24T23:44:46.238 INFO:teuthology.orchestra.run.smithi146.stdout: elfutils-libs x86_64 0.188-3.el8 CentOS-BaseOS 299 k 2023-03-24T23:44:46.238 INFO:teuthology.orchestra.run.smithi146.stdout: libblkid x86_64 2.32.1-41.el8 CentOS-BaseOS 220 k 2023-03-24T23:44:46.239 INFO:teuthology.orchestra.run.smithi146.stdout: libfdisk x86_64 2.32.1-41.el8 CentOS-BaseOS 253 k 2023-03-24T23:44:46.239 INFO:teuthology.orchestra.run.smithi146.stdout: libmount x86_64 2.32.1-41.el8 CentOS-BaseOS 236 k 2023-03-24T23:44:46.239 INFO:teuthology.orchestra.run.smithi146.stdout: librados2 x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 3.7 M 2023-03-24T23:44:46.239 INFO:teuthology.orchestra.run.smithi146.stdout: librbd1 x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 3.6 M 2023-03-24T23:44:46.239 INFO:teuthology.orchestra.run.smithi146.stdout: libsmartcols x86_64 2.32.1-41.el8 CentOS-BaseOS 179 k 2023-03-24T23:44:46.239 INFO:teuthology.orchestra.run.smithi146.stdout: libuuid x86_64 2.32.1-41.el8 CentOS-BaseOS 98 k 2023-03-24T23:44:46.239 INFO:teuthology.orchestra.run.smithi146.stdout: openssl x86_64 1:1.1.1k-9.el8 CentOS-BaseOS 737 k 2023-03-24T23:44:46.239 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-libs x86_64 1:1.1.1k-9.el8 CentOS-BaseOS 1.5 M 2023-03-24T23:44:46.240 INFO:teuthology.orchestra.run.smithi146.stdout: util-linux x86_64 2.32.1-41.el8 CentOS-BaseOS 2.6 M 2023-03-24T23:44:46.240 INFO:teuthology.orchestra.run.smithi146.stdout: util-linux-user x86_64 2.32.1-41.el8 CentOS-BaseOS 102 k 2023-03-24T23:44:46.240 INFO:teuthology.orchestra.run.smithi146.stdout:Installing dependencies: 2023-03-24T23:44:46.240 INFO:teuthology.orchestra.run.smithi146.stdout: automake noarch 1.16.1-7.el8 CentOS-AppStream 713 k 2023-03-24T23:44:46.240 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-common x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 22 M 2023-03-24T23:44:46.240 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-grafana-dashboards noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 23 k 2023-03-24T23:44:46.240 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mds x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 2.3 M 2023-03-24T23:44:46.241 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-modules-core noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 239 k 2023-03-24T23:44:46.241 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mon x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 4.6 M 2023-03-24T23:44:46.241 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-osd x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 18 M 2023-03-24T23:44:46.241 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-prometheus-alerts noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 15 k 2023-03-24T23:44:46.241 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-selinux x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 24 k 2023-03-24T23:44:46.241 INFO:teuthology.orchestra.run.smithi146.stdout: fuse x86_64 2.9.7-16.el8 CentOS-BaseOS 83 k 2023-03-24T23:44:46.241 INFO:teuthology.orchestra.run.smithi146.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-03-24T23:44:46.242 INFO:teuthology.orchestra.run.smithi146.stdout: jq x86_64 1.6-6.el8 CentOS-AppStream 202 k 2023-03-24T23:44:46.242 INFO:teuthology.orchestra.run.smithi146.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 CentOS-BaseOS 48 k 2023-03-24T23:44:46.242 INFO:teuthology.orchestra.run.smithi146.stdout: krb5-devel x86_64 1.18.2-22.el8 CentOS-BaseOS 561 k 2023-03-24T23:44:46.242 INFO:teuthology.orchestra.run.smithi146.stdout: libcephsqlite x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 137 k 2023-03-24T23:44:46.242 INFO:teuthology.orchestra.run.smithi146.stdout: libcom_err-devel x86_64 1.45.6-5.el8 CentOS-BaseOS 39 k 2023-03-24T23:44:46.243 INFO:teuthology.orchestra.run.smithi146.stdout: libgfortran x86_64 8.5.0-18.el8 CentOS-BaseOS 645 k 2023-03-24T23:44:46.243 INFO:teuthology.orchestra.run.smithi146.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-03-24T23:44:46.243 INFO:teuthology.orchestra.run.smithi146.stdout: libpq x86_64 13.5-1.el8 CentOS-AppStream 198 k 2023-03-24T23:44:46.243 INFO:teuthology.orchestra.run.smithi146.stdout: libquadmath x86_64 8.5.0-18.el8 CentOS-BaseOS 172 k 2023-03-24T23:44:46.243 INFO:teuthology.orchestra.run.smithi146.stdout: librabbitmq x86_64 0.9.0-3.el8 CentOS-BaseOS 47 k 2023-03-24T23:44:46.243 INFO:teuthology.orchestra.run.smithi146.stdout: libradosstriper1 x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 474 k 2023-03-24T23:44:46.243 INFO:teuthology.orchestra.run.smithi146.stdout: librdkafka x86_64 0.11.4-3.el8 CentOS-AppStream 354 k 2023-03-24T23:44:46.244 INFO:teuthology.orchestra.run.smithi146.stdout: librgw2 x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 6.5 M 2023-03-24T23:44:46.244 INFO:teuthology.orchestra.run.smithi146.stdout: libselinux-devel x86_64 2.9-6.el8 CentOS-BaseOS 200 k 2023-03-24T23:44:46.244 INFO:teuthology.orchestra.run.smithi146.stdout: libsepol-devel x86_64 2.9-3.el8 CentOS-BaseOS 87 k 2023-03-24T23:44:46.244 INFO:teuthology.orchestra.run.smithi146.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-03-24T23:44:46.244 INFO:teuthology.orchestra.run.smithi146.stdout: libverto-devel x86_64 0.3.2-2.el8 CentOS-BaseOS 18 k 2023-03-24T23:44:46.244 INFO:teuthology.orchestra.run.smithi146.stdout: lttng-ust x86_64 2.8.1-11.el8 CentOS-AppStream 259 k 2023-03-24T23:44:46.244 INFO:teuthology.orchestra.run.smithi146.stdout: oniguruma x86_64 6.8.2-2.el8 CentOS-AppStream 187 k 2023-03-24T23:44:46.245 INFO:teuthology.orchestra.run.smithi146.stdout: openblas x86_64 0.3.15-4.el8 CentOS-AppStream 4.7 M 2023-03-24T23:44:46.245 INFO:teuthology.orchestra.run.smithi146.stdout: openblas-threads x86_64 0.3.15-4.el8 CentOS-AppStream 4.9 M 2023-03-24T23:44:46.245 INFO:teuthology.orchestra.run.smithi146.stdout: pcre2-devel x86_64 10.32-3.el8 CentOS-BaseOS 605 k 2023-03-24T23:44:46.245 INFO:teuthology.orchestra.run.smithi146.stdout: pcre2-utf16 x86_64 10.32-3.el8 CentOS-BaseOS 229 k 2023-03-24T23:44:46.245 INFO:teuthology.orchestra.run.smithi146.stdout: pcre2-utf32 x86_64 10.32-3.el8 CentOS-BaseOS 220 k 2023-03-24T23:44:46.245 INFO:teuthology.orchestra.run.smithi146.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-03-24T23:44:46.246 INFO:teuthology.orchestra.run.smithi146.stdout: python3-babel noarch 2.5.1-7.el8 CentOS-AppStream 4.8 M 2023-03-24T23:44:46.246 INFO:teuthology.orchestra.run.smithi146.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-03-24T23:44:46.246 INFO:teuthology.orchestra.run.smithi146.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-03-24T23:44:46.246 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-03-24T23:44:46.246 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-argparse x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 45 k 2023-03-24T23:44:46.246 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-common x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 117 k 2023-03-24T23:44:46.246 INFO:teuthology.orchestra.run.smithi146.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-03-24T23:44:46.247 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cffi x86_64 1.11.5-5.el8 CentOS-BaseOS 237 k 2023-03-24T23:44:46.247 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-03-24T23:44:46.247 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-03-24T23:44:46.247 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cryptography x86_64 3.2.1-5.el8 CentOS-BaseOS 559 k 2023-03-24T23:44:46.247 INFO:teuthology.orchestra.run.smithi146.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-03-24T23:44:46.247 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-03-24T23:44:46.247 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-03-24T23:44:46.248 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jinja2 noarch 2.10.1-3.el8 CentOS-AppStream 538 k 2023-03-24T23:44:46.248 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpatch noarch 1.21-2.el8 CentOS-AppStream 27 k 2023-03-24T23:44:46.248 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpointer noarch 1.10-11.el8 CentOS-AppStream 20 k 2023-03-24T23:44:46.248 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jwt noarch 1.6.1-2.el8 CentOS-BaseOS 43 k 2023-03-24T23:44:46.248 INFO:teuthology.orchestra.run.smithi146.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-03-24T23:44:46.248 INFO:teuthology.orchestra.run.smithi146.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-03-24T23:44:46.248 INFO:teuthology.orchestra.run.smithi146.stdout: python3-mako noarch 1.0.6-14.el8 CentOS-AppStream 157 k 2023-03-24T23:44:46.249 INFO:teuthology.orchestra.run.smithi146.stdout: python3-markupsafe x86_64 0.23-19.el8 CentOS-AppStream 39 k 2023-03-24T23:44:46.249 INFO:teuthology.orchestra.run.smithi146.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-03-24T23:44:46.249 INFO:teuthology.orchestra.run.smithi146.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-03-24T23:44:46.249 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy x86_64 1:1.14.3-10.el8 CentOS-AppStream 3.7 M 2023-03-24T23:44:46.249 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 CentOS-AppStream 225 k 2023-03-24T23:44:46.249 INFO:teuthology.orchestra.run.smithi146.stdout: python3-oauthlib noarch 2.1.0-1.el8 CentOS-BaseOS 155 k 2023-03-24T23:44:46.249 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-03-24T23:44:46.250 INFO:teuthology.orchestra.run.smithi146.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-03-24T23:44:46.250 INFO:teuthology.orchestra.run.smithi146.stdout: python3-prettytable noarch 0.7.2-14.el8 CentOS-AppStream 44 k 2023-03-24T23:44:46.250 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 CentOS-AppStream 103 k 2023-03-24T23:44:46.250 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1 noarch 0.3.7-6.el8 CentOS-AppStream 126 k 2023-03-24T23:44:46.250 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 CentOS-AppStream 110 k 2023-03-24T23:44:46.251 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pycparser noarch 2.14-14.el8 CentOS-BaseOS 109 k 2023-03-24T23:44:46.251 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pytz noarch 2017.2-9.el8 CentOS-AppStream 54 k 2023-03-24T23:44:46.251 INFO:teuthology.orchestra.run.smithi146.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-03-24T23:44:46.251 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 CentOS-BaseOS 43 k 2023-03-24T23:44:46.251 INFO:teuthology.orchestra.run.smithi146.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-03-24T23:44:46.251 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-03-24T23:44:46.251 INFO:teuthology.orchestra.run.smithi146.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 CentOS-AppStream 14 M 2023-03-24T23:44:46.252 INFO:teuthology.orchestra.run.smithi146.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-03-24T23:44:46.252 INFO:teuthology.orchestra.run.smithi146.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-03-24T23:44:46.252 INFO:teuthology.orchestra.run.smithi146.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-03-24T23:44:46.252 INFO:teuthology.orchestra.run.smithi146.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-03-24T23:44:46.252 INFO:teuthology.orchestra.run.smithi146.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-03-24T23:44:46.252 INFO:teuthology.orchestra.run.smithi146.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-03-24T23:44:46.252 INFO:teuthology.orchestra.run.smithi146.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-03-24T23:44:46.252 INFO:teuthology.orchestra.run.smithi146.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-03-24T23:44:46.253 INFO:teuthology.orchestra.run.smithi146.stdout: python3-werkzeug noarch 0.12.2-4.el8 CentOS-AppStream 457 k 2023-03-24T23:44:46.253 INFO:teuthology.orchestra.run.smithi146.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-03-24T23:44:46.253 INFO:teuthology.orchestra.run.smithi146.stdout: socat x86_64 1.7.4.1-1.el8 CentOS-AppStream 323 k 2023-03-24T23:44:46.253 INFO:teuthology.orchestra.run.smithi146.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-03-24T23:44:46.253 INFO:teuthology.orchestra.run.smithi146.stdout: uuid x86_64 1.6.2-43.el8 CentOS-AppStream 64 k 2023-03-24T23:44:46.253 INFO:teuthology.orchestra.run.smithi146.stdout: xmlstarlet x86_64 1.6.1-20.el8 CentOS-AppStream 71 k 2023-03-24T23:44:46.253 INFO:teuthology.orchestra.run.smithi146.stdout: zlib-devel x86_64 1.2.11-19.el8 CentOS-BaseOS 58 k 2023-03-24T23:44:46.254 INFO:teuthology.orchestra.run.smithi146.stdout:Enabling module streams: 2023-03-24T23:44:46.254 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql 10 2023-03-24T23:44:46.254 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:44:46.254 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2023-03-24T23:44:46.254 INFO:teuthology.orchestra.run.smithi146.stdout:========================================================================================================= 2023-03-24T23:44:46.254 INFO:teuthology.orchestra.run.smithi146.stdout:Install 127 Packages 2023-03-24T23:44:46.254 INFO:teuthology.orchestra.run.smithi146.stdout:Upgrade 15 Packages 2023-03-24T23:44:46.255 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:44:46.255 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 204 M 2023-03-24T23:44:46.256 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2023-03-24T23:44:46.368 INFO:teuthology.orchestra.run.smithi146.stdout:(1/142): ceph-18.0.0-3124.gc19e8f2a.el8.x86_64. 53 kB/s | 6.4 kB 00:00 2023-03-24T23:44:46.560 INFO:teuthology.orchestra.run.smithi146.stdout:(2/142): ceph-fuse-18.0.0-3124.gc19e8f2a.el8.x8 4.3 MB/s | 846 kB 00:00 2023-03-24T23:44:46.585 INFO:teuthology.orchestra.run.smithi072.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-03-24T23:44:46.585 INFO:teuthology.orchestra.run.smithi072.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-03-24T23:44:46.598 INFO:teuthology.orchestra.run.smithi072.stdout:Package NetworkManager-1:1.39.12-1.el8.x86_64 is already installed. 2023-03-24T23:44:46.598 INFO:teuthology.orchestra.run.smithi072.stdout:Package NetworkManager-initscripts-updown-1:1.39.12-1.el8.noarch is already installed. 2023-03-24T23:44:46.599 INFO:teuthology.orchestra.run.smithi072.stdout:Package iproute-5.18.0-1.el8.x86_64 is already installed. 2023-03-24T23:44:46.600 INFO:teuthology.orchestra.run.smithi072.stdout:Package util-linux-2.32.1-38.el8.x86_64 is already installed. 2023-03-24T23:44:46.602 INFO:teuthology.orchestra.run.smithi072.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-03-24T23:44:46.603 INFO:teuthology.orchestra.run.smithi072.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-03-24T23:44:46.606 INFO:teuthology.orchestra.run.smithi072.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-03-24T23:44:46.607 INFO:teuthology.orchestra.run.smithi072.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-03-24T23:44:46.619 INFO:teuthology.orchestra.run.smithi146.stdout:(3/142): ceph-immutable-object-cache-18.0.0-312 2.8 MB/s | 165 kB 00:00 2023-03-24T23:44:46.742 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-03-24T23:44:46.755 INFO:teuthology.orchestra.run.smithi072.stdout:========================================================================================================= 2023-03-24T23:44:46.756 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2023-03-24T23:44:46.756 INFO:teuthology.orchestra.run.smithi072.stdout:========================================================================================================= 2023-03-24T23:44:46.756 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-03-24T23:44:46.757 INFO:teuthology.orchestra.run.smithi072.stdout: bison x86_64 3.0.4-10.el8 CentOS-AppStream 688 k 2023-03-24T23:44:46.757 INFO:teuthology.orchestra.run.smithi072.stdout: ceph x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 6.4 k 2023-03-24T23:44:46.757 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-base x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 5.4 M 2023-03-24T23:44:46.757 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-fuse x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 846 k 2023-03-24T23:44:46.758 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 165 k 2023-03-24T23:44:46.758 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 1.5 M 2023-03-24T23:44:46.758 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-cephadm noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 123 k 2023-03-24T23:44:46.759 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-dashboard noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 1.7 M 2023-03-24T23:44:46.759 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 7.4 M 2023-03-24T23:44:46.759 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-rook noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 47 k 2023-03-24T23:44:46.760 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-radosgw x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 13 M 2023-03-24T23:44:46.760 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-test x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 41 M 2023-03-24T23:44:46.760 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-volume noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 263 k 2023-03-24T23:44:46.761 INFO:teuthology.orchestra.run.smithi072.stdout: cephadm noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 83 k 2023-03-24T23:44:46.761 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-mirror x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 258 k 2023-03-24T23:44:46.761 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-top noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 18 k 2023-03-24T23:44:46.761 INFO:teuthology.orchestra.run.smithi072.stdout: elfutils-libelf-devel x86_64 0.188-3.el8 CentOS-BaseOS 61 k 2023-03-24T23:44:46.762 INFO:teuthology.orchestra.run.smithi072.stdout: flex x86_64 2.6.1-9.el8 CentOS-AppStream 320 k 2023-03-24T23:44:46.762 INFO:teuthology.orchestra.run.smithi072.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 19 k 2023-03-24T23:44:46.762 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs-devel x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 30 k 2023-03-24T23:44:46.762 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs2 x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 769 k 2023-03-24T23:44:46.763 INFO:teuthology.orchestra.run.smithi072.stdout: librados-devel x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 139 k 2023-03-24T23:44:46.763 INFO:teuthology.orchestra.run.smithi072.stdout: libtool x86_64 2.4.6-25.el8 CentOS-AppStream 709 k 2023-03-24T23:44:46.763 INFO:teuthology.orchestra.run.smithi072.stdout: libuuid-devel x86_64 2.32.1-41.el8 CentOS-BaseOS 99 k 2023-03-24T23:44:46.763 INFO:teuthology.orchestra.run.smithi072.stdout: openssl-devel x86_64 1:1.1.1k-9.el8 CentOS-BaseOS 3.2 M 2023-03-24T23:44:46.764 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql x86_64 10.23-1.module_el8+300+2c53004c CentOS-AppStream 1.6 M 2023-03-24T23:44:46.764 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-contrib x86_64 10.23-1.module_el8+300+2c53004c CentOS-AppStream 903 k 2023-03-24T23:44:46.764 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-server x86_64 10.23-1.module_el8+300+2c53004c CentOS-AppStream 5.1 M 2023-03-24T23:44:46.764 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cephfs x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 190 k 2023-03-24T23:44:46.765 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rados x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 370 k 2023-03-24T23:44:46.765 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rbd x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 364 k 2023-03-24T23:44:46.765 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rgw x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 107 k 2023-03-24T23:44:46.766 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-fuse x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 94 k 2023-03-24T23:44:46.766 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-mirror x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 3.4 M 2023-03-24T23:44:46.766 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-nbd x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 179 k 2023-03-24T23:44:46.766 INFO:teuthology.orchestra.run.smithi072.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 CentOS-BaseOS 72 k 2023-03-24T23:44:46.766 INFO:teuthology.orchestra.run.smithi072.stdout:Upgrading: 2023-03-24T23:44:46.767 INFO:teuthology.orchestra.run.smithi072.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-1.el8 CentOS-BaseOS 142 k 2023-03-24T23:44:46.767 INFO:teuthology.orchestra.run.smithi072.stdout: elfutils-debuginfod-client x86_64 0.188-3.el8 CentOS-BaseOS 75 k 2023-03-24T23:44:46.767 INFO:teuthology.orchestra.run.smithi072.stdout: elfutils-libelf x86_64 0.188-3.el8 CentOS-BaseOS 231 k 2023-03-24T23:44:46.767 INFO:teuthology.orchestra.run.smithi072.stdout: elfutils-libs x86_64 0.188-3.el8 CentOS-BaseOS 299 k 2023-03-24T23:44:46.768 INFO:teuthology.orchestra.run.smithi072.stdout: libblkid x86_64 2.32.1-41.el8 CentOS-BaseOS 220 k 2023-03-24T23:44:46.768 INFO:teuthology.orchestra.run.smithi072.stdout: libfdisk x86_64 2.32.1-41.el8 CentOS-BaseOS 253 k 2023-03-24T23:44:46.768 INFO:teuthology.orchestra.run.smithi072.stdout: libmount x86_64 2.32.1-41.el8 CentOS-BaseOS 236 k 2023-03-24T23:44:46.768 INFO:teuthology.orchestra.run.smithi072.stdout: librados2 x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 3.7 M 2023-03-24T23:44:46.769 INFO:teuthology.orchestra.run.smithi072.stdout: librbd1 x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 3.6 M 2023-03-24T23:44:46.769 INFO:teuthology.orchestra.run.smithi072.stdout: libsmartcols x86_64 2.32.1-41.el8 CentOS-BaseOS 179 k 2023-03-24T23:44:46.769 INFO:teuthology.orchestra.run.smithi072.stdout: libuuid x86_64 2.32.1-41.el8 CentOS-BaseOS 98 k 2023-03-24T23:44:46.769 INFO:teuthology.orchestra.run.smithi072.stdout: openssl x86_64 1:1.1.1k-9.el8 CentOS-BaseOS 737 k 2023-03-24T23:44:46.770 INFO:teuthology.orchestra.run.smithi072.stdout: openssl-libs x86_64 1:1.1.1k-9.el8 CentOS-BaseOS 1.5 M 2023-03-24T23:44:46.770 INFO:teuthology.orchestra.run.smithi072.stdout: util-linux x86_64 2.32.1-41.el8 CentOS-BaseOS 2.6 M 2023-03-24T23:44:46.770 INFO:teuthology.orchestra.run.smithi072.stdout: util-linux-user x86_64 2.32.1-41.el8 CentOS-BaseOS 102 k 2023-03-24T23:44:46.770 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2023-03-24T23:44:46.771 INFO:teuthology.orchestra.run.smithi072.stdout: automake noarch 1.16.1-7.el8 CentOS-AppStream 713 k 2023-03-24T23:44:46.771 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-common x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 22 M 2023-03-24T23:44:46.771 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-grafana-dashboards noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 23 k 2023-03-24T23:44:46.771 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mds x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 2.3 M 2023-03-24T23:44:46.772 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-modules-core noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 239 k 2023-03-24T23:44:46.772 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mon x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 4.6 M 2023-03-24T23:44:46.772 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-osd x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 18 M 2023-03-24T23:44:46.772 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-prometheus-alerts noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 15 k 2023-03-24T23:44:46.772 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-selinux x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 24 k 2023-03-24T23:44:46.772 INFO:teuthology.orchestra.run.smithi072.stdout: fuse x86_64 2.9.7-16.el8 CentOS-BaseOS 83 k 2023-03-24T23:44:46.773 INFO:teuthology.orchestra.run.smithi072.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-03-24T23:44:46.773 INFO:teuthology.orchestra.run.smithi072.stdout: jq x86_64 1.6-6.el8 CentOS-AppStream 202 k 2023-03-24T23:44:46.773 INFO:teuthology.orchestra.run.smithi072.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 CentOS-BaseOS 48 k 2023-03-24T23:44:46.773 INFO:teuthology.orchestra.run.smithi072.stdout: krb5-devel x86_64 1.18.2-22.el8 CentOS-BaseOS 561 k 2023-03-24T23:44:46.773 INFO:teuthology.orchestra.run.smithi072.stdout: libcephsqlite x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 137 k 2023-03-24T23:44:46.774 INFO:teuthology.orchestra.run.smithi072.stdout: libcom_err-devel x86_64 1.45.6-5.el8 CentOS-BaseOS 39 k 2023-03-24T23:44:46.774 INFO:teuthology.orchestra.run.smithi072.stdout: libgfortran x86_64 8.5.0-18.el8 CentOS-BaseOS 645 k 2023-03-24T23:44:46.774 INFO:teuthology.orchestra.run.smithi072.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-03-24T23:44:46.775 INFO:teuthology.orchestra.run.smithi072.stdout: libpq x86_64 13.5-1.el8 CentOS-AppStream 198 k 2023-03-24T23:44:46.775 INFO:teuthology.orchestra.run.smithi072.stdout: libquadmath x86_64 8.5.0-18.el8 CentOS-BaseOS 172 k 2023-03-24T23:44:46.775 INFO:teuthology.orchestra.run.smithi072.stdout: librabbitmq x86_64 0.9.0-3.el8 CentOS-BaseOS 47 k 2023-03-24T23:44:46.775 INFO:teuthology.orchestra.run.smithi072.stdout: libradosstriper1 x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 474 k 2023-03-24T23:44:46.776 INFO:teuthology.orchestra.run.smithi072.stdout: librdkafka x86_64 0.11.4-3.el8 CentOS-AppStream 354 k 2023-03-24T23:44:46.776 INFO:teuthology.orchestra.run.smithi072.stdout: librgw2 x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 6.5 M 2023-03-24T23:44:46.776 INFO:teuthology.orchestra.run.smithi072.stdout: libselinux-devel x86_64 2.9-6.el8 CentOS-BaseOS 200 k 2023-03-24T23:44:46.776 INFO:teuthology.orchestra.run.smithi072.stdout: libsepol-devel x86_64 2.9-3.el8 CentOS-BaseOS 87 k 2023-03-24T23:44:46.777 INFO:teuthology.orchestra.run.smithi072.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-03-24T23:44:46.777 INFO:teuthology.orchestra.run.smithi072.stdout: libverto-devel x86_64 0.3.2-2.el8 CentOS-BaseOS 18 k 2023-03-24T23:44:46.777 INFO:teuthology.orchestra.run.smithi072.stdout: lttng-ust x86_64 2.8.1-11.el8 CentOS-AppStream 259 k 2023-03-24T23:44:46.777 INFO:teuthology.orchestra.run.smithi072.stdout: oniguruma x86_64 6.8.2-2.el8 CentOS-AppStream 187 k 2023-03-24T23:44:46.777 INFO:teuthology.orchestra.run.smithi072.stdout: openblas x86_64 0.3.15-4.el8 CentOS-AppStream 4.7 M 2023-03-24T23:44:46.778 INFO:teuthology.orchestra.run.smithi072.stdout: openblas-threads x86_64 0.3.15-4.el8 CentOS-AppStream 4.9 M 2023-03-24T23:44:46.778 INFO:teuthology.orchestra.run.smithi072.stdout: pcre2-devel x86_64 10.32-3.el8 CentOS-BaseOS 605 k 2023-03-24T23:44:46.778 INFO:teuthology.orchestra.run.smithi072.stdout: pcre2-utf16 x86_64 10.32-3.el8 CentOS-BaseOS 229 k 2023-03-24T23:44:46.778 INFO:teuthology.orchestra.run.smithi072.stdout: pcre2-utf32 x86_64 10.32-3.el8 CentOS-BaseOS 220 k 2023-03-24T23:44:46.778 INFO:teuthology.orchestra.run.smithi072.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-03-24T23:44:46.779 INFO:teuthology.orchestra.run.smithi072.stdout: python3-babel noarch 2.5.1-7.el8 CentOS-AppStream 4.8 M 2023-03-24T23:44:46.779 INFO:teuthology.orchestra.run.smithi072.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-03-24T23:44:46.779 INFO:teuthology.orchestra.run.smithi072.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-03-24T23:44:46.779 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-03-24T23:44:46.779 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-argparse x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 45 k 2023-03-24T23:44:46.779 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-common x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 117 k 2023-03-24T23:44:46.779 INFO:teuthology.orchestra.run.smithi072.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-03-24T23:44:46.779 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cffi x86_64 1.11.5-5.el8 CentOS-BaseOS 237 k 2023-03-24T23:44:46.780 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-03-24T23:44:46.780 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-03-24T23:44:46.780 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cryptography x86_64 3.2.1-5.el8 CentOS-BaseOS 559 k 2023-03-24T23:44:46.780 INFO:teuthology.orchestra.run.smithi072.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-03-24T23:44:46.780 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-03-24T23:44:46.780 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-03-24T23:44:46.780 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jinja2 noarch 2.10.1-3.el8 CentOS-AppStream 538 k 2023-03-24T23:44:46.780 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpatch noarch 1.21-2.el8 CentOS-AppStream 27 k 2023-03-24T23:44:46.781 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpointer noarch 1.10-11.el8 CentOS-AppStream 20 k 2023-03-24T23:44:46.781 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jwt noarch 1.6.1-2.el8 CentOS-BaseOS 43 k 2023-03-24T23:44:46.781 INFO:teuthology.orchestra.run.smithi072.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-03-24T23:44:46.781 INFO:teuthology.orchestra.run.smithi072.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-03-24T23:44:46.781 INFO:teuthology.orchestra.run.smithi072.stdout: python3-mako noarch 1.0.6-14.el8 CentOS-AppStream 157 k 2023-03-24T23:44:46.781 INFO:teuthology.orchestra.run.smithi072.stdout: python3-markupsafe x86_64 0.23-19.el8 CentOS-AppStream 39 k 2023-03-24T23:44:46.781 INFO:teuthology.orchestra.run.smithi072.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-03-24T23:44:46.782 INFO:teuthology.orchestra.run.smithi072.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-03-24T23:44:46.782 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy x86_64 1:1.14.3-10.el8 CentOS-AppStream 3.7 M 2023-03-24T23:44:46.782 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 CentOS-AppStream 225 k 2023-03-24T23:44:46.782 INFO:teuthology.orchestra.run.smithi072.stdout: python3-oauthlib noarch 2.1.0-1.el8 CentOS-BaseOS 155 k 2023-03-24T23:44:46.782 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-03-24T23:44:46.783 INFO:teuthology.orchestra.run.smithi072.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-03-24T23:44:46.783 INFO:teuthology.orchestra.run.smithi072.stdout: python3-prettytable noarch 0.7.2-14.el8 CentOS-AppStream 44 k 2023-03-24T23:44:46.783 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 CentOS-AppStream 103 k 2023-03-24T23:44:46.783 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1 noarch 0.3.7-6.el8 CentOS-AppStream 126 k 2023-03-24T23:44:46.783 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 CentOS-AppStream 110 k 2023-03-24T23:44:46.783 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pycparser noarch 2.14-14.el8 CentOS-BaseOS 109 k 2023-03-24T23:44:46.783 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pytz noarch 2017.2-9.el8 CentOS-AppStream 54 k 2023-03-24T23:44:46.783 INFO:teuthology.orchestra.run.smithi072.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-03-24T23:44:46.784 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 CentOS-BaseOS 43 k 2023-03-24T23:44:46.784 INFO:teuthology.orchestra.run.smithi072.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-03-24T23:44:46.784 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-03-24T23:44:46.784 INFO:teuthology.orchestra.run.smithi072.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 CentOS-AppStream 14 M 2023-03-24T23:44:46.784 INFO:teuthology.orchestra.run.smithi072.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-03-24T23:44:46.784 INFO:teuthology.orchestra.run.smithi072.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-03-24T23:44:46.784 INFO:teuthology.orchestra.run.smithi072.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-03-24T23:44:46.785 INFO:teuthology.orchestra.run.smithi072.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-03-24T23:44:46.785 INFO:teuthology.orchestra.run.smithi072.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-03-24T23:44:46.785 INFO:teuthology.orchestra.run.smithi072.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-03-24T23:44:46.785 INFO:teuthology.orchestra.run.smithi072.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-03-24T23:44:46.785 INFO:teuthology.orchestra.run.smithi072.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-03-24T23:44:46.785 INFO:teuthology.orchestra.run.smithi072.stdout: python3-werkzeug noarch 0.12.2-4.el8 CentOS-AppStream 457 k 2023-03-24T23:44:46.786 INFO:teuthology.orchestra.run.smithi072.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-03-24T23:44:46.786 INFO:teuthology.orchestra.run.smithi072.stdout: socat x86_64 1.7.4.1-1.el8 CentOS-AppStream 323 k 2023-03-24T23:44:46.786 INFO:teuthology.orchestra.run.smithi072.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-03-24T23:44:46.786 INFO:teuthology.orchestra.run.smithi072.stdout: uuid x86_64 1.6.2-43.el8 CentOS-AppStream 64 k 2023-03-24T23:44:46.786 INFO:teuthology.orchestra.run.smithi072.stdout: xmlstarlet x86_64 1.6.1-20.el8 CentOS-AppStream 71 k 2023-03-24T23:44:46.786 INFO:teuthology.orchestra.run.smithi072.stdout: zlib-devel x86_64 1.2.11-19.el8 CentOS-BaseOS 58 k 2023-03-24T23:44:46.786 INFO:teuthology.orchestra.run.smithi072.stdout:Enabling module streams: 2023-03-24T23:44:46.786 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql 10 2023-03-24T23:44:46.787 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:44:46.787 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-03-24T23:44:46.787 INFO:teuthology.orchestra.run.smithi072.stdout:========================================================================================================= 2023-03-24T23:44:46.787 INFO:teuthology.orchestra.run.smithi072.stdout:Install 127 Packages 2023-03-24T23:44:46.787 INFO:teuthology.orchestra.run.smithi072.stdout:Upgrade 15 Packages 2023-03-24T23:44:46.787 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:44:46.788 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 204 M 2023-03-24T23:44:46.788 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-03-24T23:44:46.950 INFO:teuthology.orchestra.run.smithi072.stdout:(1/142): ceph-18.0.0-3124.gc19e8f2a.el8.x86_64. 37 kB/s | 6.4 kB 00:00 2023-03-24T23:44:47.008 INFO:teuthology.orchestra.run.smithi146.stdout:(4/142): ceph-base-18.0.0-3124.gc19e8f2a.el8.x8 7.1 MB/s | 5.4 MB 00:00 2023-03-24T23:44:47.061 INFO:teuthology.orchestra.run.smithi146.stdout:(5/142): ceph-mds-18.0.0-3124.gc19e8f2a.el8.x86 5.1 MB/s | 2.3 MB 00:00 2023-03-24T23:44:47.253 INFO:teuthology.orchestra.run.smithi146.stdout:(6/142): ceph-mgr-18.0.0-3124.gc19e8f2a.el8.x86 6.3 MB/s | 1.5 MB 00:00 2023-03-24T23:44:47.367 INFO:teuthology.orchestra.run.smithi072.stdout:(2/142): ceph-fuse-18.0.0-3124.gc19e8f2a.el8.x8 2.0 MB/s | 846 kB 00:00 2023-03-24T23:44:47.468 INFO:teuthology.orchestra.run.smithi072.stdout:(3/142): ceph-immutable-object-cache-18.0.0-312 1.6 MB/s | 165 kB 00:00 2023-03-24T23:44:48.086 INFO:teuthology.orchestra.run.smithi146.stdout:(7/142): ceph-mon-18.0.0-3124.gc19e8f2a.el8.x86 4.5 MB/s | 4.6 MB 00:01 2023-03-24T23:44:48.493 INFO:teuthology.orchestra.run.smithi072.stdout:(4/142): ceph-base-18.0.0-3124.gc19e8f2a.el8.x8 3.1 MB/s | 5.4 MB 00:01 2023-03-24T23:44:48.794 INFO:teuthology.orchestra.run.smithi072.stdout:(5/142): ceph-mds-18.0.0-3124.gc19e8f2a.el8.x86 1.7 MB/s | 2.3 MB 00:01 2023-03-24T23:44:49.011 INFO:teuthology.orchestra.run.smithi072.stdout:(6/142): ceph-mgr-18.0.0-3124.gc19e8f2a.el8.x86 3.0 MB/s | 1.5 MB 00:00 2023-03-24T23:44:49.255 INFO:teuthology.orchestra.run.smithi146.stdout:(8/142): ceph-common-18.0.0-3124.gc19e8f2a.el8. 7.2 MB/s | 22 MB 00:03 2023-03-24T23:44:49.328 INFO:teuthology.orchestra.run.smithi146.stdout:(9/142): ceph-selinux-18.0.0-3124.gc19e8f2a.el8 325 kB/s | 24 kB 00:00 2023-03-24T23:44:50.295 INFO:teuthology.orchestra.run.smithi146.stdout:(10/142): ceph-osd-18.0.0-3124.gc19e8f2a.el8.x8 5.8 MB/s | 18 MB 00:03 2023-03-24T23:44:50.354 INFO:teuthology.orchestra.run.smithi146.stdout:(11/142): cephfs-mirror-18.0.0-3124.gc19e8f2a.e 4.3 MB/s | 258 kB 00:00 2023-03-24T23:44:50.404 INFO:teuthology.orchestra.run.smithi146.stdout:(12/142): libcephfs-devel-18.0.0-3124.gc19e8f2a 609 kB/s | 30 kB 00:00 2023-03-24T23:44:50.546 INFO:teuthology.orchestra.run.smithi146.stdout:(13/142): libcephfs2-18.0.0-3124.gc19e8f2a.el8. 5.3 MB/s | 769 kB 00:00 2023-03-24T23:44:50.595 INFO:teuthology.orchestra.run.smithi072.stdout:(7/142): ceph-common-18.0.0-3124.gc19e8f2a.el8. 5.7 MB/s | 22 MB 00:03 2023-03-24T23:44:50.613 INFO:teuthology.orchestra.run.smithi146.stdout:(14/142): libcephsqlite-18.0.0-3124.gc19e8f2a.e 2.0 MB/s | 137 kB 00:00 2023-03-24T23:44:50.688 INFO:teuthology.orchestra.run.smithi146.stdout:(15/142): librados-devel-18.0.0-3124.gc19e8f2a. 1.8 MB/s | 139 kB 00:00 2023-03-24T23:44:50.695 INFO:teuthology.orchestra.run.smithi072.stdout:(8/142): ceph-mon-18.0.0-3124.gc19e8f2a.el8.x86 2.4 MB/s | 4.6 MB 00:01 2023-03-24T23:44:50.779 INFO:teuthology.orchestra.run.smithi072.stdout:(9/142): ceph-selinux-18.0.0-3124.gc19e8f2a.el8 292 kB/s | 24 kB 00:00 2023-03-24T23:44:50.838 INFO:teuthology.orchestra.run.smithi146.stdout:(16/142): libradosstriper1-18.0.0-3124.gc19e8f2 3.1 MB/s | 474 kB 00:00 2023-03-24T23:44:50.972 INFO:teuthology.orchestra.run.smithi146.stdout:(17/142): ceph-radosgw-18.0.0-3124.gc19e8f2a.el 4.5 MB/s | 13 MB 00:02 2023-03-24T23:44:51.064 INFO:teuthology.orchestra.run.smithi146.stdout:(18/142): python3-ceph-argparse-18.0.0-3124.gc1 487 kB/s | 45 kB 00:00 2023-03-24T23:44:51.156 INFO:teuthology.orchestra.run.smithi146.stdout:(19/142): python3-ceph-common-18.0.0-3124.gc19e 1.2 MB/s | 117 kB 00:00 2023-03-24T23:44:51.248 INFO:teuthology.orchestra.run.smithi146.stdout:(20/142): python3-cephfs-18.0.0-3124.gc19e8f2a. 2.0 MB/s | 190 kB 00:00 2023-03-24T23:44:51.424 INFO:teuthology.orchestra.run.smithi146.stdout:(21/142): python3-rados-18.0.0-3124.gc19e8f2a.e 2.1 MB/s | 370 kB 00:00 2023-03-24T23:44:51.591 INFO:teuthology.orchestra.run.smithi146.stdout:(22/142): python3-rbd-18.0.0-3124.gc19e8f2a.el8 2.1 MB/s | 364 kB 00:00 2023-03-24T23:44:51.691 INFO:teuthology.orchestra.run.smithi146.stdout:(23/142): python3-rgw-18.0.0-3124.gc19e8f2a.el8 1.0 MB/s | 107 kB 00:00 2023-03-24T23:44:51.774 INFO:teuthology.orchestra.run.smithi146.stdout:(24/142): rbd-fuse-18.0.0-3124.gc19e8f2a.el8.x8 1.1 MB/s | 94 kB 00:00 2023-03-24T23:44:52.150 INFO:teuthology.orchestra.run.smithi146.stdout:(25/142): librgw2-18.0.0-3124.gc19e8f2a.el8.x86 5.0 MB/s | 6.5 MB 00:01 2023-03-24T23:44:52.250 INFO:teuthology.orchestra.run.smithi146.stdout:(26/142): rbd-nbd-18.0.0-3124.gc19e8f2a.el8.x86 1.8 MB/s | 179 kB 00:00 2023-03-24T23:44:52.308 INFO:teuthology.orchestra.run.smithi146.stdout:(27/142): ceph-grafana-dashboards-18.0.0-3124.g 396 kB/s | 23 kB 00:00 2023-03-24T23:44:52.367 INFO:teuthology.orchestra.run.smithi146.stdout:(28/142): ceph-mgr-cephadm-18.0.0-3124.gc19e8f2 2.1 MB/s | 123 kB 00:00 2023-03-24T23:44:52.421 INFO:teuthology.orchestra.run.smithi072.stdout:(10/142): ceph-radosgw-18.0.0-3124.gc19e8f2a.el 7.2 MB/s | 13 MB 00:01 2023-03-24T23:44:52.484 INFO:teuthology.orchestra.run.smithi146.stdout:(29/142): rbd-mirror-18.0.0-3124.gc19e8f2a.el8. 4.9 MB/s | 3.4 MB 00:00 2023-03-24T23:44:52.505 INFO:teuthology.orchestra.run.smithi072.stdout:(11/142): cephfs-mirror-18.0.0-3124.gc19e8f2a.e 3.0 MB/s | 258 kB 00:00 2023-03-24T23:44:52.589 INFO:teuthology.orchestra.run.smithi072.stdout:(12/142): libcephfs-devel-18.0.0-3124.gc19e8f2a 365 kB/s | 30 kB 00:00 2023-03-24T23:44:52.668 INFO:teuthology.orchestra.run.smithi146.stdout:(30/142): ceph-mgr-dashboard-18.0.0-3124.gc19e8 5.6 MB/s | 1.7 MB 00:00 2023-03-24T23:44:52.739 INFO:teuthology.orchestra.run.smithi072.stdout:(13/142): libcephfs2-18.0.0-3124.gc19e8f2a.el8. 5.0 MB/s | 769 kB 00:00 2023-03-24T23:44:52.760 INFO:teuthology.orchestra.run.smithi146.stdout:(31/142): ceph-mgr-modules-core-18.0.0-3124.gc1 2.6 MB/s | 239 kB 00:00 2023-03-24T23:44:52.815 INFO:teuthology.orchestra.run.smithi072.stdout:(14/142): libcephsqlite-18.0.0-3124.gc19e8f2a.e 1.8 MB/s | 137 kB 00:00 2023-03-24T23:44:52.827 INFO:teuthology.orchestra.run.smithi146.stdout:(32/142): ceph-mgr-rook-18.0.0-3124.gc19e8f2a.e 709 kB/s | 47 kB 00:00 2023-03-24T23:44:52.885 INFO:teuthology.orchestra.run.smithi146.stdout:(33/142): ceph-prometheus-alerts-18.0.0-3124.gc 249 kB/s | 15 kB 00:00 2023-03-24T23:44:52.952 INFO:teuthology.orchestra.run.smithi146.stdout:(34/142): ceph-volume-18.0.0-3124.gc19e8f2a.el8 3.9 MB/s | 263 kB 00:00 2023-03-24T23:44:53.019 INFO:teuthology.orchestra.run.smithi146.stdout:(35/142): cephadm-18.0.0-3124.gc19e8f2a.el8.noa 1.2 MB/s | 83 kB 00:00 2023-03-24T23:44:53.069 INFO:teuthology.orchestra.run.smithi146.stdout:(36/142): cephfs-top-18.0.0-3124.gc19e8f2a.el8. 361 kB/s | 18 kB 00:00 2023-03-24T23:44:53.116 INFO:teuthology.orchestra.run.smithi072.stdout:(15/142): ceph-osd-18.0.0-3124.gc19e8f2a.el8.x8 4.3 MB/s | 18 MB 00:04 2023-03-24T23:44:53.141 INFO:teuthology.orchestra.run.smithi072.stdout:(16/142): librados-devel-18.0.0-3124.gc19e8f2a. 426 kB/s | 139 kB 00:00 2023-03-24T23:44:53.161 INFO:teuthology.orchestra.run.smithi146.stdout:(37/142): automake-1.16.1-7.el8.noarch.rpm 7.6 MB/s | 713 kB 00:00 2023-03-24T23:44:53.195 INFO:teuthology.orchestra.run.smithi146.stdout:(38/142): bison-3.0.4-10.el8.x86_64.rpm 20 MB/s | 688 kB 00:00 2023-03-24T23:44:53.221 INFO:teuthology.orchestra.run.smithi146.stdout:(39/142): flex-2.6.1-9.el8.x86_64.rpm 12 MB/s | 320 kB 00:00 2023-03-24T23:44:53.235 INFO:teuthology.orchestra.run.smithi072.stdout:(17/142): libradosstriper1-18.0.0-3124.gc19e8f2 3.9 MB/s | 474 kB 00:00 2023-03-24T23:44:53.246 INFO:teuthology.orchestra.run.smithi146.stdout:(40/142): jq-1.6-6.el8.x86_64.rpm 7.9 MB/s | 202 kB 00:00 2023-03-24T23:44:53.271 INFO:teuthology.orchestra.run.smithi146.stdout:(41/142): libpq-13.5-1.el8.x86_64.rpm 7.8 MB/s | 198 kB 00:00 2023-03-24T23:44:53.302 INFO:teuthology.orchestra.run.smithi072.stdout:(18/142): python3-ceph-argparse-18.0.0-3124.gc1 666 kB/s | 45 kB 00:00 2023-03-24T23:44:53.305 INFO:teuthology.orchestra.run.smithi146.stdout:(42/142): librdkafka-0.11.4-3.el8.x86_64.rpm 10 MB/s | 354 kB 00:00 2023-03-24T23:44:53.338 INFO:teuthology.orchestra.run.smithi146.stdout:(43/142): libtool-2.4.6-25.el8.x86_64.rpm 21 MB/s | 709 kB 00:00 2023-03-24T23:44:53.361 INFO:teuthology.orchestra.run.smithi072.stdout:(19/142): python3-ceph-common-18.0.0-3124.gc19e 1.9 MB/s | 117 kB 00:00 2023-03-24T23:44:53.364 INFO:teuthology.orchestra.run.smithi146.stdout:(44/142): lttng-ust-2.8.1-11.el8.x86_64.rpm 10 MB/s | 259 kB 00:00 2023-03-24T23:44:53.389 INFO:teuthology.orchestra.run.smithi146.stdout:(45/142): oniguruma-6.8.2-2.el8.x86_64.rpm 7.3 MB/s | 187 kB 00:00 2023-03-24T23:44:53.428 INFO:teuthology.orchestra.run.smithi072.stdout:(20/142): python3-cephfs-18.0.0-3124.gc19e8f2a. 2.8 MB/s | 190 kB 00:00 2023-03-24T23:44:53.498 INFO:teuthology.orchestra.run.smithi146.stdout:(46/142): openblas-0.3.15-4.el8.x86_64.rpm 44 MB/s | 4.7 MB 00:00 2023-03-24T23:44:53.554 INFO:teuthology.orchestra.run.smithi072.stdout:(21/142): python3-rados-18.0.0-3124.gc19e8f2a.e 2.9 MB/s | 370 kB 00:00 2023-03-24T23:44:53.607 INFO:teuthology.orchestra.run.smithi146.stdout:(47/142): openblas-threads-0.3.15-4.el8.x86_64. 45 MB/s | 4.9 MB 00:00 2023-03-24T23:44:53.657 INFO:teuthology.orchestra.run.smithi146.stdout:(48/142): postgresql-10.23-1.module_el8+300+2c5 31 MB/s | 1.6 MB 00:00 2023-03-24T23:44:53.679 INFO:teuthology.orchestra.run.smithi072.stdout:(22/142): python3-rbd-18.0.0-3124.gc19e8f2a.el8 2.8 MB/s | 364 kB 00:00 2023-03-24T23:44:53.690 INFO:teuthology.orchestra.run.smithi146.stdout:(49/142): postgresql-contrib-10.23-1.module_el8 26 MB/s | 903 kB 00:00 2023-03-24T23:44:53.772 INFO:teuthology.orchestra.run.smithi072.stdout:(23/142): python3-rgw-18.0.0-3124.gc19e8f2a.el8 1.1 MB/s | 107 kB 00:00 2023-03-24T23:44:53.783 INFO:teuthology.orchestra.run.smithi146.stdout:(50/142): ceph-mgr-diskprediction-local-18.0.0- 5.7 MB/s | 7.4 MB 00:01 2023-03-24T23:44:53.883 INFO:teuthology.orchestra.run.smithi146.stdout:(51/142): postgresql-server-10.23-1.module_el8+ 27 MB/s | 5.1 MB 00:00 2023-03-24T23:44:53.897 INFO:teuthology.orchestra.run.smithi072.stdout:(24/142): librgw2-18.0.0-3124.gc19e8f2a.el8.x86 8.6 MB/s | 6.5 MB 00:00 2023-03-24T23:44:53.916 INFO:teuthology.orchestra.run.smithi146.stdout:(52/142): python3-jinja2-2.10.1-3.el8.noarch.rp 16 MB/s | 538 kB 00:00 2023-03-24T23:44:53.923 INFO:teuthology.orchestra.run.smithi072.stdout:(25/142): rbd-fuse-18.0.0-3124.gc19e8f2a.el8.x8 624 kB/s | 94 kB 00:00 2023-03-24T23:44:53.933 INFO:teuthology.orchestra.run.smithi146.stdout:(53/142): python3-jsonpatch-1.21-2.el8.noarch.r 1.6 MB/s | 27 kB 00:00 2023-03-24T23:44:53.950 INFO:teuthology.orchestra.run.smithi146.stdout:(54/142): python3-jsonpointer-1.10-11.el8.noarc 1.2 MB/s | 20 kB 00:00 2023-03-24T23:44:53.984 INFO:teuthology.orchestra.run.smithi146.stdout:(55/142): python3-mako-1.0.6-14.el8.noarch.rpm 4.6 MB/s | 157 kB 00:00 2023-03-24T23:44:53.999 INFO:teuthology.orchestra.run.smithi072.stdout:(26/142): rbd-nbd-18.0.0-3124.gc19e8f2a.el8.x86 2.3 MB/s | 179 kB 00:00 2023-03-24T23:44:54.001 INFO:teuthology.orchestra.run.smithi146.stdout:(56/142): python3-markupsafe-0.23-19.el8.x86_64 2.3 MB/s | 39 kB 00:00 2023-03-24T23:44:54.091 INFO:teuthology.orchestra.run.smithi072.stdout:(27/142): ceph-grafana-dashboards-18.0.0-3124.g 251 kB/s | 23 kB 00:00 2023-03-24T23:44:54.093 INFO:teuthology.orchestra.run.smithi146.stdout:(57/142): python3-numpy-1.14.3-10.el8.x86_64.rp 41 MB/s | 3.7 MB 00:00 2023-03-24T23:44:54.118 INFO:teuthology.orchestra.run.smithi146.stdout:(58/142): python3-numpy-f2py-1.14.3-10.el8.x86_ 8.8 MB/s | 225 kB 00:00 2023-03-24T23:44:54.135 INFO:teuthology.orchestra.run.smithi146.stdout:(59/142): python3-prettytable-0.7.2-14.el8.noar 2.6 MB/s | 44 kB 00:00 2023-03-24T23:44:54.158 INFO:teuthology.orchestra.run.smithi072.stdout:(28/142): ceph-mgr-cephadm-18.0.0-3124.gc19e8f2 1.8 MB/s | 123 kB 00:00 2023-03-24T23:44:54.194 INFO:teuthology.orchestra.run.smithi146.stdout:(60/142): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.7 MB/s | 103 kB 00:00 2023-03-24T23:44:54.219 INFO:teuthology.orchestra.run.smithi146.stdout:(61/142): python3-pyasn1-0.3.7-6.el8.noarch.rpm 4.9 MB/s | 126 kB 00:00 2023-03-24T23:44:54.244 INFO:teuthology.orchestra.run.smithi146.stdout:(62/142): python3-pyasn1-modules-0.3.7-6.el8.no 4.3 MB/s | 110 kB 00:00 2023-03-24T23:44:54.250 INFO:teuthology.orchestra.run.smithi072.stdout:(29/142): rbd-mirror-18.0.0-3124.gc19e8f2a.el8. 9.8 MB/s | 3.4 MB 00:00 2023-03-24T23:44:54.427 INFO:teuthology.orchestra.run.smithi072.stdout:(30/142): ceph-mgr-dashboard-18.0.0-3124.gc19e8 6.3 MB/s | 1.7 MB 00:00 2023-03-24T23:44:54.503 INFO:teuthology.orchestra.run.smithi072.stdout:(31/142): ceph-mgr-modules-core-18.0.0-3124.gc1 3.1 MB/s | 239 kB 00:00 2023-03-24T23:44:54.571 INFO:teuthology.orchestra.run.smithi072.stdout:(32/142): ceph-mgr-rook-18.0.0-3124.gc19e8f2a.e 707 kB/s | 47 kB 00:00 2023-03-24T23:44:54.603 INFO:teuthology.orchestra.run.smithi146.stdout:(63/142): ceph-test-18.0.0-3124.gc19e8f2a.el8.x 7.7 MB/s | 41 MB 00:05 2023-03-24T23:44:54.628 INFO:teuthology.orchestra.run.smithi146.stdout:(64/142): python3-pytz-2017.2-9.el8.noarch.rpm 141 kB/s | 54 kB 00:00 2023-03-24T23:44:54.629 INFO:teuthology.orchestra.run.smithi072.stdout:(33/142): ceph-prometheus-alerts-18.0.0-3124.gc 248 kB/s | 15 kB 00:00 2023-03-24T23:44:54.679 INFO:teuthology.orchestra.run.smithi146.stdout:(65/142): python3-werkzeug-0.12.2-4.el8.noarch. 9.8 MB/s | 457 kB 00:00 2023-03-24T23:44:54.705 INFO:teuthology.orchestra.run.smithi072.stdout:(34/142): ceph-volume-18.0.0-3124.gc19e8f2a.el8 3.4 MB/s | 263 kB 00:00 2023-03-24T23:44:54.721 INFO:teuthology.orchestra.run.smithi146.stdout:(66/142): socat-1.7.4.1-1.el8.x86_64.rpm 8.0 MB/s | 323 kB 00:00 2023-03-24T23:44:54.796 INFO:teuthology.orchestra.run.smithi146.stdout:(67/142): python3-babel-2.5.1-7.el8.noarch.rpm 4.7 MB/s | 4.8 MB 00:01 2023-03-24T23:44:54.821 INFO:teuthology.orchestra.run.smithi146.stdout:(68/142): uuid-1.6.2-43.el8.x86_64.rpm 638 kB/s | 64 kB 00:00 2023-03-24T23:44:54.847 INFO:teuthology.orchestra.run.smithi146.stdout:(69/142): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 71 kB 00:00 2023-03-24T23:44:54.872 INFO:teuthology.orchestra.run.smithi072.stdout:(35/142): ceph-mgr-diskprediction-local-18.0.0- 12 MB/s | 7.4 MB 00:00 2023-03-24T23:44:54.905 INFO:teuthology.orchestra.run.smithi146.stdout:(70/142): elfutils-libelf-devel-0.188-3.el8.x86 732 kB/s | 61 kB 00:00 2023-03-24T23:44:54.906 INFO:teuthology.orchestra.run.smithi072.stdout:(36/142): cephadm-18.0.0-3124.gc19e8f2a.el8.noa 413 kB/s | 83 kB 00:00 2023-03-24T23:44:54.931 INFO:teuthology.orchestra.run.smithi146.stdout:(71/142): fuse-2.9.7-16.el8.x86_64.rpm 990 kB/s | 83 kB 00:00 2023-03-24T23:44:54.931 INFO:teuthology.orchestra.run.smithi072.stdout:(37/142): cephfs-top-18.0.0-3124.gc19e8f2a.el8. 306 kB/s | 18 kB 00:00 2023-03-24T23:44:54.956 INFO:teuthology.orchestra.run.smithi146.stdout:(72/142): keyutils-libs-devel-1.5.10-9.el8.x86_ 961 kB/s | 48 kB 00:00 2023-03-24T23:44:54.981 INFO:teuthology.orchestra.run.smithi146.stdout:(73/142): libaio-devel-0.3.112-1.el8.x86_64.rpm 750 kB/s | 19 kB 00:00 2023-03-24T23:44:55.007 INFO:teuthology.orchestra.run.smithi072.stdout:(38/142): automake-1.16.1-7.el8.noarch.rpm 6.9 MB/s | 713 kB 00:00 2023-03-24T23:44:55.015 INFO:teuthology.orchestra.run.smithi146.stdout:(74/142): libcom_err-devel-1.45.6-5.el8.x86_64. 1.1 MB/s | 39 kB 00:00 2023-03-24T23:44:55.049 INFO:teuthology.orchestra.run.smithi146.stdout:(75/142): krb5-devel-1.18.2-22.el8.x86_64.rpm 4.7 MB/s | 561 kB 00:00 2023-03-24T23:44:55.049 INFO:teuthology.orchestra.run.smithi072.stdout:(39/142): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-03-24T23:44:55.082 INFO:teuthology.orchestra.run.smithi146.stdout:(76/142): libquadmath-8.5.0-18.el8.x86_64.rpm 5.1 MB/s | 172 kB 00:00 2023-03-24T23:44:55.117 INFO:teuthology.orchestra.run.smithi072.stdout:(40/142): jq-1.6-6.el8.x86_64.rpm 3.0 MB/s | 202 kB 00:00 2023-03-24T23:44:55.124 INFO:teuthology.orchestra.run.smithi146.stdout:(77/142): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2023-03-24T23:44:55.158 INFO:teuthology.orchestra.run.smithi146.stdout:(78/142): libgfortran-8.5.0-18.el8.x86_64.rpm 4.4 MB/s | 645 kB 00:00 2023-03-24T23:44:55.159 INFO:teuthology.orchestra.run.smithi072.stdout:(41/142): bison-3.0.4-10.el8.x86_64.rpm 3.0 MB/s | 688 kB 00:00 2023-03-24T23:44:55.185 INFO:teuthology.orchestra.run.smithi072.stdout:(42/142): libpq-13.5-1.el8.x86_64.rpm 2.9 MB/s | 198 kB 00:00 2023-03-24T23:44:55.208 INFO:teuthology.orchestra.run.smithi146.stdout:(79/142): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-03-24T23:44:55.227 INFO:teuthology.orchestra.run.smithi072.stdout:(43/142): librdkafka-0.11.4-3.el8.x86_64.rpm 5.1 MB/s | 354 kB 00:00 2023-03-24T23:44:55.242 INFO:teuthology.orchestra.run.smithi146.stdout:(80/142): libselinux-devel-2.9-6.el8.x86_64.rpm 1.7 MB/s | 200 kB 00:00 2023-03-24T23:44:55.259 INFO:teuthology.orchestra.run.smithi146.stdout:(81/142): libuuid-devel-2.32.1-41.el8.x86_64.rp 1.9 MB/s | 99 kB 00:00 2023-03-24T23:44:55.261 INFO:teuthology.orchestra.run.smithi072.stdout:(44/142): libtool-2.4.6-25.el8.x86_64.rpm 9.1 MB/s | 709 kB 00:00 2023-03-24T23:44:55.284 INFO:teuthology.orchestra.run.smithi146.stdout:(82/142): libverto-devel-0.3.2-2.el8.x86_64.rpm 431 kB/s | 18 kB 00:00 2023-03-24T23:44:55.296 INFO:teuthology.orchestra.run.smithi072.stdout:(45/142): oniguruma-6.8.2-2.el8.x86_64.rpm 5.2 MB/s | 187 kB 00:00 2023-03-24T23:44:55.322 INFO:teuthology.orchestra.run.smithi072.stdout:(46/142): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.7 MB/s | 259 kB 00:00 2023-03-24T23:44:55.334 INFO:teuthology.orchestra.run.smithi146.stdout:(83/142): pcre2-devel-10.32-3.el8.x86_64.rpm 12 MB/s | 605 kB 00:00 2023-03-24T23:44:55.476 INFO:teuthology.orchestra.run.smithi146.stdout:(84/142): python3-scipy-1.0.0-21.module_el8.5.0 16 MB/s | 14 MB 00:00 2023-03-24T23:44:55.489 INFO:teuthology.orchestra.run.smithi072.stdout:(47/142): openblas-0.3.15-4.el8.x86_64.rpm 25 MB/s | 4.7 MB 00:00 2023-03-24T23:44:55.501 INFO:teuthology.orchestra.run.smithi146.stdout:(85/142): pcre2-utf16-10.32-3.el8.x86_64.rpm 1.3 MB/s | 229 kB 00:00 2023-03-24T23:44:55.527 INFO:teuthology.orchestra.run.smithi146.stdout:(86/142): pcre2-utf32-10.32-3.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-03-24T23:44:55.560 INFO:teuthology.orchestra.run.smithi146.stdout:(87/142): python3-cryptography-3.2.1-5.el8.x86_ 17 MB/s | 559 kB 00:00 2023-03-24T23:44:55.565 INFO:teuthology.orchestra.run.smithi072.stdout:(48/142): postgresql-10.23-1.module_el8+300+2c5 21 MB/s | 1.6 MB 00:00 2023-03-24T23:44:55.594 INFO:teuthology.orchestra.run.smithi146.stdout:(88/142): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.5 MB/s | 237 kB 00:00 2023-03-24T23:44:55.611 INFO:teuthology.orchestra.run.smithi146.stdout:(89/142): python3-jwt-1.6.1-2.el8.noarch.rpm 846 kB/s | 43 kB 00:00 2023-03-24T23:44:55.665 INFO:teuthology.orchestra.run.smithi072.stdout:(49/142): openblas-threads-0.3.15-4.el8.x86_64. 14 MB/s | 4.9 MB 00:00 2023-03-24T23:44:55.669 INFO:teuthology.orchestra.run.smithi146.stdout:(90/142): openssl-devel-1.1.1k-9.el8.x86_64.rpm 7.7 MB/s | 3.2 MB 00:00 2023-03-24T23:44:55.694 INFO:teuthology.orchestra.run.smithi146.stdout:(91/142): python3-pycparser-2.14-14.el8.noarch. 1.3 MB/s | 109 kB 00:00 2023-03-24T23:44:55.708 INFO:teuthology.orchestra.run.smithi072.stdout:(50/142): postgresql-contrib-10.23-1.module_el8 6.2 MB/s | 903 kB 00:00 2023-03-24T23:44:55.711 INFO:teuthology.orchestra.run.smithi146.stdout:(92/142): python3-oauthlib-2.1.0-1.el8.noarch.r 1.3 MB/s | 155 kB 00:00 2023-03-24T23:44:55.737 INFO:teuthology.orchestra.run.smithi146.stdout:(93/142): python3-requests-oauthlib-1.0.0-1.el8 1.0 MB/s | 43 kB 00:00 2023-03-24T23:44:55.762 INFO:teuthology.orchestra.run.smithi146.stdout:(94/142): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.1 MB/s | 72 kB 00:00 2023-03-24T23:44:55.788 INFO:teuthology.orchestra.run.smithi146.stdout:(95/142): zlib-devel-1.2.11-19.el8.x86_64.rpm 764 kB/s | 58 kB 00:00 2023-03-24T23:44:55.875 INFO:teuthology.orchestra.run.smithi072.stdout:(51/142): python3-babel-2.5.1-7.el8.noarch.rpm 29 MB/s | 4.8 MB 00:00 2023-03-24T23:44:55.904 INFO:teuthology.orchestra.run.smithi146.stdout:(96/142): liboath-2.6.2-3.el8.x86_64.rpm 414 kB/s | 59 kB 00:00 2023-03-24T23:44:55.917 INFO:teuthology.orchestra.run.smithi072.stdout:(52/142): python3-jinja2-2.10.1-3.el8.noarch.rp 13 MB/s | 538 kB 00:00 2023-03-24T23:44:55.930 INFO:teuthology.orchestra.run.smithi146.stdout:(97/142): libunwind-1.3.1-3.el8.x86_64.rpm 528 kB/s | 75 kB 00:00 2023-03-24T23:44:55.988 INFO:teuthology.orchestra.run.smithi146.stdout:(98/142): gperftools-libs-2.7-9.el8.x86_64.rpm 1.2 MB/s | 306 kB 00:00 2023-03-24T23:44:56.013 INFO:teuthology.orchestra.run.smithi146.stdout:(99/142): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 522 kB/s | 44 kB 00:00 2023-03-24T23:44:56.017 INFO:teuthology.orchestra.run.smithi072.stdout:(53/142): postgresql-server-10.23-1.module_el8+ 15 MB/s | 5.1 MB 00:00 2023-03-24T23:44:56.039 INFO:teuthology.orchestra.run.smithi146.stdout:(100/142): python3-beautifulsoup4-4.6.3-2.el8.1 3.6 MB/s | 185 kB 00:00 2023-03-24T23:44:56.043 INFO:teuthology.orchestra.run.smithi072.stdout:(54/142): python3-jsonpatch-1.21-2.el8.noarch.r 211 kB/s | 27 kB 00:00 2023-03-24T23:44:56.064 INFO:teuthology.orchestra.run.smithi146.stdout:(101/142): python3-cachetools-3.1.1-4.el8.noarc 657 kB/s | 33 kB 00:00 2023-03-24T23:44:56.070 INFO:teuthology.orchestra.run.smithi072.stdout:(55/142): python3-jsonpointer-1.10-11.el8.noarc 392 kB/s | 20 kB 00:00 2023-03-24T23:44:56.089 INFO:teuthology.orchestra.run.smithi146.stdout:(102/142): python3-certifi-2018.10.15-7.el8.noa 324 kB/s | 16 kB 00:00 2023-03-24T23:44:56.103 INFO:teuthology.orchestra.run.smithi072.stdout:(56/142): python3-mako-1.0.6-14.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-03-24T23:44:56.115 INFO:teuthology.orchestra.run.smithi146.stdout:(103/142): python3-asyncssh-2.7.0-2.el8.noarch. 2.2 MB/s | 476 kB 00:00 2023-03-24T23:44:56.129 INFO:teuthology.orchestra.run.smithi072.stdout:(57/142): python3-markupsafe-0.23-19.el8.x86_64 653 kB/s | 39 kB 00:00 2023-03-24T23:44:56.162 INFO:teuthology.orchestra.run.smithi072.stdout:(58/142): python3-numpy-f2py-1.14.3-10.el8.x86_ 6.6 MB/s | 225 kB 00:00 2023-03-24T23:44:56.165 INFO:teuthology.orchestra.run.smithi146.stdout:(104/142): python3-cheroot-8.5.2-1.el8.noarch.r 1.7 MB/s | 173 kB 00:00 2023-03-24T23:44:56.198 INFO:teuthology.orchestra.run.smithi072.stdout:(59/142): python3-prettytable-0.7.2-14.el8.noar 1.2 MB/s | 44 kB 00:00 2023-03-24T23:44:56.209 INFO:teuthology.orchestra.run.smithi146.stdout:(105/142): python3-google-auth-1.1.1-10.el8.noa 865 kB/s | 81 kB 00:00 2023-03-24T23:44:56.223 INFO:teuthology.orchestra.run.smithi072.stdout:(60/142): python3-pyOpenSSL-19.0.0-1.el8.noarch 4.0 MB/s | 103 kB 00:00 2023-03-24T23:44:56.241 INFO:teuthology.orchestra.run.smithi146.stdout:(106/142): python3-jaraco-6.2-6.el8.noarch.rpm 147 kB/s | 11 kB 00:00 2023-03-24T23:44:56.257 INFO:teuthology.orchestra.run.smithi072.stdout:(61/142): python3-pyasn1-0.3.7-6.el8.noarch.rpm 3.7 MB/s | 126 kB 00:00 2023-03-24T23:44:56.266 INFO:teuthology.orchestra.run.smithi146.stdout:(107/142): python3-jaraco-functools-2.0-4.el8.n 309 kB/s | 18 kB 00:00 2023-03-24T23:44:56.283 INFO:teuthology.orchestra.run.smithi072.stdout:(62/142): python3-pyasn1-modules-0.3.7-6.el8.no 4.2 MB/s | 110 kB 00:00 2023-03-24T23:44:56.291 INFO:teuthology.orchestra.run.smithi146.stdout:(108/142): python3-cherrypy-18.4.0-1.el8.noarch 1.9 MB/s | 384 kB 00:00 2023-03-24T23:44:56.309 INFO:teuthology.orchestra.run.smithi072.stdout:(63/142): python3-pytz-2017.2-9.el8.noarch.rpm 2.1 MB/s | 54 kB 00:00 2023-03-24T23:44:56.316 INFO:teuthology.orchestra.run.smithi146.stdout:(109/142): python3-logutils-0.3.5-11.el8.noarch 971 kB/s | 49 kB 00:00 2023-03-24T23:44:56.342 INFO:teuthology.orchestra.run.smithi146.stdout:(110/142): python3-more-itertools-7.2.0-3.el8.n 1.1 MB/s | 59 kB 00:00 2023-03-24T23:44:56.367 INFO:teuthology.orchestra.run.smithi146.stdout:(111/142): python3-natsort-7.1.1-2.el8.noarch.r 1.2 MB/s | 60 kB 00:00 2023-03-24T23:44:56.417 INFO:teuthology.orchestra.run.smithi146.stdout:(112/142): python3-portend-2.6-1.el8.noarch.rpm 312 kB/s | 16 kB 00:00 2023-03-24T23:44:56.418 INFO:teuthology.orchestra.run.smithi072.stdout:(64/142): python3-numpy-1.14.3-10.el8.x86_64.rp 12 MB/s | 3.7 MB 00:00 2023-03-24T23:44:56.443 INFO:teuthology.orchestra.run.smithi146.stdout:(113/142): python3-pecan-1.3.2-9.el8.noarch.rpm 2.7 MB/s | 283 kB 00:00 2023-03-24T23:44:56.460 INFO:teuthology.orchestra.run.smithi072.stdout:(65/142): python3-werkzeug-0.12.2-4.el8.noarch. 11 MB/s | 457 kB 00:00 2023-03-24T23:44:56.476 INFO:teuthology.orchestra.run.smithi146.stdout:(114/142): python3-kubernetes-11.0.0-6.el8.noar 5.0 MB/s | 1.2 MB 00:00 2023-03-24T23:44:56.493 INFO:teuthology.orchestra.run.smithi072.stdout:(66/142): socat-1.7.4.1-1.el8.x86_64.rpm 9.4 MB/s | 323 kB 00:00 2023-03-24T23:44:56.501 INFO:teuthology.orchestra.run.smithi146.stdout:(115/142): python3-repoze-lru-0.7-6.el8.noarch. 401 kB/s | 34 kB 00:00 2023-03-24T23:44:56.527 INFO:teuthology.orchestra.run.smithi146.stdout:(116/142): python3-routes-2.4.1-12.el8.noarch.r 2.3 MB/s | 196 kB 00:00 2023-03-24T23:44:56.528 INFO:teuthology.orchestra.run.smithi072.stdout:(67/142): uuid-1.6.2-43.el8.x86_64.rpm 1.9 MB/s | 64 kB 00:00 2023-03-24T23:44:56.552 INFO:teuthology.orchestra.run.smithi146.stdout:(117/142): python3-rsa-4.8-1.el8.noarch.rpm 815 kB/s | 61 kB 00:00 2023-03-24T23:44:56.553 INFO:teuthology.orchestra.run.smithi072.stdout:(68/142): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 71 kB 00:00 2023-03-24T23:44:56.577 INFO:teuthology.orchestra.run.smithi146.stdout:(118/142): python3-simplegeneric-0.8.1-17.el8.n 247 kB/s | 19 kB 00:00 2023-03-24T23:44:56.587 INFO:teuthology.orchestra.run.smithi072.stdout:(69/142): elfutils-libelf-devel-0.188-3.el8.x86 1.8 MB/s | 61 kB 00:00 2023-03-24T23:44:56.603 INFO:teuthology.orchestra.run.smithi146.stdout:(119/142): python3-singledispatch-3.4.0.3-18.el 317 kB/s | 24 kB 00:00 2023-03-24T23:44:56.613 INFO:teuthology.orchestra.run.smithi072.stdout:(70/142): fuse-2.9.7-16.el8.x86_64.rpm 3.2 MB/s | 83 kB 00:00 2023-03-24T23:44:56.628 INFO:teuthology.orchestra.run.smithi146.stdout:(120/142): python3-tempora-1.14.1-5.el8.noarch. 388 kB/s | 29 kB 00:00 2023-03-24T23:44:56.638 INFO:teuthology.orchestra.run.smithi072.stdout:(71/142): keyutils-libs-devel-1.5.10-9.el8.x86_ 1.9 MB/s | 48 kB 00:00 2023-03-24T23:44:56.653 INFO:teuthology.orchestra.run.smithi146.stdout:(121/142): python3-trustme-0.6.0-4.el8.noarch.r 364 kB/s | 27 kB 00:00 2023-03-24T23:44:56.679 INFO:teuthology.orchestra.run.smithi146.stdout:(122/142): python3-waitress-1.2.1-2.el8.1.noarc 3.1 MB/s | 241 kB 00:00 2023-03-24T23:44:56.680 INFO:teuthology.orchestra.run.smithi072.stdout:(72/142): krb5-devel-1.18.2-22.el8.x86_64.rpm 13 MB/s | 561 kB 00:00 2023-03-24T23:44:56.704 INFO:teuthology.orchestra.run.smithi146.stdout:(123/142): python3-webob-1.8.5-1.el8.1.noarch.r 3.2 MB/s | 251 kB 00:00 2023-03-24T23:44:56.708 INFO:teuthology.orchestra.run.smithi072.stdout:(73/142): libaio-devel-0.3.112-1.el8.x86_64.rpm 693 kB/s | 19 kB 00:00 2023-03-24T23:44:56.729 INFO:teuthology.orchestra.run.smithi146.stdout:(124/142): python3-websocket-client-0.56.0-5.el 809 kB/s | 61 kB 00:00 2023-03-24T23:44:56.733 INFO:teuthology.orchestra.run.smithi072.stdout:(74/142): libcom_err-devel-1.45.6-5.el8.x86_64. 1.5 MB/s | 39 kB 00:00 2023-03-24T23:44:56.754 INFO:teuthology.orchestra.run.smithi146.stdout:(125/142): python3-webtest-2.0.33-1.el8.noarch. 1.1 MB/s | 85 kB 00:00 2023-03-24T23:44:56.775 INFO:teuthology.orchestra.run.smithi072.stdout:(75/142): libgfortran-8.5.0-18.el8.x86_64.rpm 15 MB/s | 645 kB 00:00 2023-03-24T23:44:56.779 INFO:teuthology.orchestra.run.smithi146.stdout:(126/142): python3-zc-lockfile-2.0-2.el8.noarch 303 kB/s | 23 kB 00:00 2023-03-24T23:44:56.809 INFO:teuthology.orchestra.run.smithi072.stdout:(76/142): libquadmath-8.5.0-18.el8.x86_64.rpm 5.0 MB/s | 172 kB 00:00 2023-03-24T23:44:56.835 INFO:teuthology.orchestra.run.smithi072.stdout:(77/142): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.8 MB/s | 47 kB 00:00 2023-03-24T23:44:56.869 INFO:teuthology.orchestra.run.smithi072.stdout:(78/142): libselinux-devel-2.9-6.el8.x86_64.rpm 5.8 MB/s | 200 kB 00:00 2023-03-24T23:44:56.894 INFO:teuthology.orchestra.run.smithi072.stdout:(79/142): libsepol-devel-2.9-3.el8.x86_64.rpm 3.4 MB/s | 87 kB 00:00 2023-03-24T23:44:56.920 INFO:teuthology.orchestra.run.smithi072.stdout:(80/142): libuuid-devel-2.32.1-41.el8.x86_64.rp 3.9 MB/s | 99 kB 00:00 2023-03-24T23:44:56.945 INFO:teuthology.orchestra.run.smithi072.stdout:(81/142): libverto-devel-0.3.2-2.el8.x86_64.rpm 712 kB/s | 18 kB 00:00 2023-03-24T23:44:57.071 INFO:teuthology.orchestra.run.smithi072.stdout:(82/142): openssl-devel-1.1.1k-9.el8.x86_64.rpm 25 MB/s | 3.2 MB 00:00 2023-03-24T23:44:57.097 INFO:teuthology.orchestra.run.smithi146.stdout:(127/142): thrift-0.13.0-2.el8.x86_64.rpm 4.7 MB/s | 1.7 MB 00:00 2023-03-24T23:44:57.122 INFO:teuthology.orchestra.run.smithi072.stdout:(83/142): pcre2-devel-10.32-3.el8.x86_64.rpm 12 MB/s | 605 kB 00:00 2023-03-24T23:44:57.130 INFO:teuthology.orchestra.run.smithi146.stdout:(128/142): NetworkManager-initscripts-updown-1. 4.2 MB/s | 142 kB 00:00 2023-03-24T23:44:57.155 INFO:teuthology.orchestra.run.smithi072.stdout:(84/142): pcre2-utf16-10.32-3.el8.x86_64.rpm 6.7 MB/s | 229 kB 00:00 2023-03-24T23:44:57.164 INFO:teuthology.orchestra.run.smithi146.stdout:(129/142): elfutils-debuginfod-client-0.188-3.e 2.2 MB/s | 75 kB 00:00 2023-03-24T23:44:57.189 INFO:teuthology.orchestra.run.smithi072.stdout:(85/142): pcre2-utf32-10.32-3.el8.x86_64.rpm 6.4 MB/s | 220 kB 00:00 2023-03-24T23:44:57.223 INFO:teuthology.orchestra.run.smithi072.stdout:(86/142): python3-cffi-1.11.5-5.el8.x86_64.rpm 6.9 MB/s | 237 kB 00:00 2023-03-24T23:44:57.267 INFO:teuthology.orchestra.run.smithi072.stdout:(87/142): python3-cryptography-3.2.1-5.el8.x86_ 13 MB/s | 559 kB 00:00 2023-03-24T23:44:57.272 INFO:teuthology.orchestra.run.smithi146.stdout:(130/142): librbd1-18.0.0-3124.gc19e8f2a.el8.x8 7.2 MB/s | 3.6 MB 00:00 2023-03-24T23:44:57.293 INFO:teuthology.orchestra.run.smithi072.stdout:(88/142): python3-jwt-1.6.1-2.el8.noarch.rpm 1.6 MB/s | 43 kB 00:00 2023-03-24T23:44:57.298 INFO:teuthology.orchestra.run.smithi146.stdout:(131/142): elfutils-libelf-0.188-3.el8.x86_64.r 1.7 MB/s | 231 kB 00:00 2023-03-24T23:44:57.327 INFO:teuthology.orchestra.run.smithi072.stdout:(89/142): python3-oauthlib-2.1.0-1.el8.noarch.r 4.5 MB/s | 155 kB 00:00 2023-03-24T23:44:57.352 INFO:teuthology.orchestra.run.smithi072.stdout:(90/142): python3-pycparser-2.14-14.el8.noarch. 4.2 MB/s | 109 kB 00:00 2023-03-24T23:44:57.364 INFO:teuthology.orchestra.run.smithi146.stdout:(132/142): librados2-18.0.0-3124.gc19e8f2a.el8. 6.0 MB/s | 3.7 MB 00:00 2023-03-24T23:44:57.378 INFO:teuthology.orchestra.run.smithi072.stdout:(91/142): python3-requests-oauthlib-1.0.0-1.el8 1.7 MB/s | 43 kB 00:00 2023-03-24T23:44:57.390 INFO:teuthology.orchestra.run.smithi146.stdout:(133/142): libblkid-2.32.1-41.el8.x86_64.rpm 2.3 MB/s | 220 kB 00:00 2023-03-24T23:44:57.404 INFO:teuthology.orchestra.run.smithi072.stdout:(92/142): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 2.8 MB/s | 72 kB 00:00 2023-03-24T23:44:57.424 INFO:teuthology.orchestra.run.smithi146.stdout:(134/142): elfutils-libs-0.188-3.el8.x86_64.rpm 1.9 MB/s | 299 kB 00:00 2023-03-24T23:44:57.429 INFO:teuthology.orchestra.run.smithi072.stdout:(93/142): zlib-devel-1.2.11-19.el8.x86_64.rpm 2.2 MB/s | 58 kB 00:00 2023-03-24T23:44:57.449 INFO:teuthology.orchestra.run.smithi146.stdout:(135/142): libmount-2.32.1-41.el8.x86_64.rpm 3.9 MB/s | 236 kB 00:00 2023-03-24T23:44:57.474 INFO:teuthology.orchestra.run.smithi146.stdout:(136/142): libfdisk-2.32.1-41.el8.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2023-03-24T23:44:57.508 INFO:teuthology.orchestra.run.smithi146.stdout:(137/142): libsmartcols-2.32.1-41.el8.x86_64.rp 2.1 MB/s | 179 kB 00:00 2023-03-24T23:44:57.525 INFO:teuthology.orchestra.run.smithi146.stdout:(138/142): libuuid-2.32.1-41.el8.x86_64.rpm 1.3 MB/s | 98 kB 00:00 2023-03-24T23:44:57.575 INFO:teuthology.orchestra.run.smithi146.stdout:(139/142): openssl-1.1.1k-9.el8.x86_64.rpm 7.2 MB/s | 737 kB 00:00 2023-03-24T23:44:57.601 INFO:teuthology.orchestra.run.smithi146.stdout:(140/142): util-linux-user-2.32.1-41.el8.x86_64 4.0 MB/s | 102 kB 00:00 2023-03-24T23:44:57.650 INFO:teuthology.orchestra.run.smithi146.stdout:(141/142): openssl-libs-1.1.1k-9.el8.x86_64.rpm 10 MB/s | 1.5 MB 00:00 2023-03-24T23:44:57.655 INFO:teuthology.orchestra.run.smithi072.stdout:(94/142): python3-scipy-1.0.0-21.module_el8.5.0 10 MB/s | 14 MB 00:01 2023-03-24T23:44:57.709 INFO:teuthology.orchestra.run.smithi146.stdout:(142/142): util-linux-2.32.1-41.el8.x86_64.rpm 14 MB/s | 2.6 MB 00:00 2023-03-24T23:44:57.713 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2023-03-24T23:44:57.714 INFO:teuthology.orchestra.run.smithi146.stdout:Total 18 MB/s | 204 MB 00:11 2023-03-24T23:44:57.789 INFO:teuthology.orchestra.run.smithi072.stdout:(95/142): liboath-2.6.2-3.el8.x86_64.rpm 440 kB/s | 59 kB 00:00 2023-03-24T23:44:57.839 INFO:teuthology.orchestra.run.smithi072.stdout:(96/142): gperftools-libs-2.7-9.el8.x86_64.rpm 747 kB/s | 306 kB 00:00 2023-03-24T23:44:57.873 INFO:teuthology.orchestra.run.smithi072.stdout:(97/142): libunwind-1.3.1-3.el8.x86_64.rpm 893 kB/s | 75 kB 00:00 2023-03-24T23:44:57.932 INFO:teuthology.orchestra.run.smithi072.stdout:(98/142): python3-asyncssh-2.7.0-2.el8.noarch.r 5.0 MB/s | 476 kB 00:00 2023-03-24T23:44:57.957 INFO:teuthology.orchestra.run.smithi072.stdout:(99/142): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 519 kB/s | 44 kB 00:00 2023-03-24T23:44:57.991 INFO:teuthology.orchestra.run.smithi072.stdout:(100/142): python3-beautifulsoup4-4.6.3-2.el8.1 3.1 MB/s | 185 kB 00:00 2023-03-24T23:44:58.018 INFO:teuthology.orchestra.run.smithi072.stdout:(101/142): python3-cachetools-3.1.1-4.el8.noarc 543 kB/s | 33 kB 00:00 2023-03-24T23:44:58.044 INFO:teuthology.orchestra.run.smithi072.stdout:(102/142): python3-certifi-2018.10.15-7.el8.noa 312 kB/s | 16 kB 00:00 2023-03-24T23:44:58.095 INFO:teuthology.orchestra.run.smithi072.stdout:(103/142): python3-cheroot-8.5.2-1.el8.noarch.r 2.2 MB/s | 173 kB 00:00 2023-03-24T23:44:58.137 INFO:teuthology.orchestra.run.smithi072.stdout:(104/142): python3-cherrypy-18.4.0-1.el8.noarch 4.1 MB/s | 384 kB 00:00 2023-03-24T23:44:58.162 INFO:teuthology.orchestra.run.smithi072.stdout:(105/142): python3-google-auth-1.1.1-10.el8.noa 1.2 MB/s | 81 kB 00:00 2023-03-24T23:44:58.188 INFO:teuthology.orchestra.run.smithi072.stdout:(106/142): python3-jaraco-6.2-6.el8.noarch.rpm 218 kB/s | 11 kB 00:00 2023-03-24T23:44:58.213 INFO:teuthology.orchestra.run.smithi072.stdout:(107/142): python3-jaraco-functools-2.0-4.el8.n 346 kB/s | 18 kB 00:00 2023-03-24T23:44:58.264 INFO:teuthology.orchestra.run.smithi072.stdout:(108/142): python3-logutils-0.3.5-11.el8.noarch 971 kB/s | 49 kB 00:00 2023-03-24T23:44:58.315 INFO:teuthology.orchestra.run.smithi072.stdout:(109/142): python3-more-itertools-7.2.0-3.el8.n 1.1 MB/s | 59 kB 00:00 2023-03-24T23:44:58.365 INFO:teuthology.orchestra.run.smithi072.stdout:(110/142): python3-natsort-7.1.1-2.el8.noarch.r 1.2 MB/s | 60 kB 00:00 2023-03-24T23:44:58.433 INFO:teuthology.orchestra.run.smithi072.stdout:(111/142): python3-kubernetes-11.0.0-6.el8.noar 4.8 MB/s | 1.2 MB 00:00 2023-03-24T23:44:58.466 INFO:teuthology.orchestra.run.smithi072.stdout:(112/142): python3-pecan-1.3.2-9.el8.noarch.rpm 2.7 MB/s | 283 kB 00:00 2023-03-24T23:44:58.492 INFO:teuthology.orchestra.run.smithi072.stdout:(113/142): python3-portend-2.6-1.el8.noarch.rpm 264 kB/s | 16 kB 00:00 2023-03-24T23:44:58.518 INFO:teuthology.orchestra.run.smithi072.stdout:(114/142): python3-repoze-lru-0.7-6.el8.noarch. 661 kB/s | 34 kB 00:00 2023-03-24T23:44:58.633 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2023-03-24T23:44:58.899 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2023-03-24T23:44:58.900 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2023-03-24T23:44:59.250 INFO:teuthology.orchestra.run.smithi072.stdout:(115/142): ceph-test-18.0.0-3124.gc19e8f2a.el8. 4.8 MB/s | 41 MB 00:08 2023-03-24T23:44:59.279 INFO:teuthology.orchestra.run.smithi072.stdout:(116/142): python3-rsa-4.8-1.el8.noarch.rpm 81 kB/s | 61 kB 00:00 2023-03-24T23:44:59.304 INFO:teuthology.orchestra.run.smithi072.stdout:(117/142): python3-routes-2.4.1-12.el8.noarch.r 242 kB/s | 196 kB 00:00 2023-03-24T23:44:59.330 INFO:teuthology.orchestra.run.smithi072.stdout:(118/142): python3-singledispatch-3.4.0.3-18.el 474 kB/s | 24 kB 00:00 2023-03-24T23:44:59.364 INFO:teuthology.orchestra.run.smithi072.stdout:(119/142): python3-simplegeneric-0.8.1-17.el8.n 165 kB/s | 19 kB 00:00 2023-03-24T23:44:59.389 INFO:teuthology.orchestra.run.smithi072.stdout:(120/142): python3-tempora-1.14.1-5.el8.noarch. 345 kB/s | 29 kB 00:00 2023-03-24T23:44:59.416 INFO:teuthology.orchestra.run.smithi072.stdout:(121/142): python3-trustme-0.6.0-4.el8.noarch.r 325 kB/s | 27 kB 00:00 2023-03-24T23:44:59.499 INFO:teuthology.orchestra.run.smithi072.stdout:(122/142): python3-websocket-client-0.56.0-5.el 732 kB/s | 61 kB 00:00 2023-03-24T23:44:59.533 INFO:teuthology.orchestra.run.smithi072.stdout:(123/142): python3-webob-1.8.5-1.el8.1.noarch.r 1.7 MB/s | 251 kB 00:00 2023-03-24T23:44:59.558 INFO:teuthology.orchestra.run.smithi072.stdout:(124/142): python3-waitress-1.2.1-2.el8.1.noarc 1.2 MB/s | 241 kB 00:00 2023-03-24T23:44:59.584 INFO:teuthology.orchestra.run.smithi072.stdout:(125/142): python3-webtest-2.0.33-1.el8.noarch. 1.0 MB/s | 85 kB 00:00 2023-03-24T23:44:59.609 INFO:teuthology.orchestra.run.smithi072.stdout:(126/142): python3-zc-lockfile-2.0-2.el8.noarch 301 kB/s | 23 kB 00:00 2023-03-24T23:44:59.826 INFO:teuthology.orchestra.run.smithi072.stdout:(127/142): thrift-0.13.0-2.el8.x86_64.rpm 6.5 MB/s | 1.7 MB 00:00 2023-03-24T23:44:59.860 INFO:teuthology.orchestra.run.smithi072.stdout:(128/142): NetworkManager-initscripts-updown-1. 4.2 MB/s | 142 kB 00:00 2023-03-24T23:44:59.886 INFO:teuthology.orchestra.run.smithi072.stdout:(129/142): elfutils-debuginfod-client-0.188-3.e 2.9 MB/s | 75 kB 00:00 2023-03-24T23:44:59.914 INFO:teuthology.orchestra.run.smithi072.stdout:(130/142): elfutils-libelf-0.188-3.el8.x86_64.r 9.0 MB/s | 231 kB 00:00 2023-03-24T23:44:59.988 INFO:teuthology.orchestra.run.smithi072.stdout:(131/142): elfutils-libs-0.188-3.el8.x86_64.rpm 3.8 MB/s | 299 kB 00:00 2023-03-24T23:45:00.022 INFO:teuthology.orchestra.run.smithi072.stdout:(132/142): libblkid-2.32.1-41.el8.x86_64.rpm 6.4 MB/s | 220 kB 00:00 2023-03-24T23:45:00.056 INFO:teuthology.orchestra.run.smithi072.stdout:(133/142): libfdisk-2.32.1-41.el8.x86_64.rpm 7.4 MB/s | 253 kB 00:00 2023-03-24T23:45:00.090 INFO:teuthology.orchestra.run.smithi072.stdout:(134/142): libmount-2.32.1-41.el8.x86_64.rpm 7.0 MB/s | 236 kB 00:00 2023-03-24T23:45:00.166 INFO:teuthology.orchestra.run.smithi072.stdout:(135/142): librbd1-18.0.0-3124.gc19e8f2a.el8.x8 6.4 MB/s | 3.6 MB 00:00 2023-03-24T23:45:00.241 INFO:teuthology.orchestra.run.smithi072.stdout:(136/142): librados2-18.0.0-3124.gc19e8f2a.el8. 5.6 MB/s | 3.7 MB 00:00 2023-03-24T23:45:00.267 INFO:teuthology.orchestra.run.smithi072.stdout:(137/142): libsmartcols-2.32.1-41.el8.x86_64.rp 1.0 MB/s | 179 kB 00:00 2023-03-24T23:45:00.292 INFO:teuthology.orchestra.run.smithi072.stdout:(138/142): libuuid-2.32.1-41.el8.x86_64.rpm 781 kB/s | 98 kB 00:00 2023-03-24T23:45:00.368 INFO:teuthology.orchestra.run.smithi072.stdout:(139/142): openssl-libs-1.1.1k-9.el8.x86_64.rpm 21 MB/s | 1.5 MB 00:00 2023-03-24T23:45:00.393 INFO:teuthology.orchestra.run.smithi072.stdout:(140/142): util-linux-user-2.32.1-41.el8.x86_64 4.0 MB/s | 102 kB 00:00 2023-03-24T23:45:00.444 INFO:teuthology.orchestra.run.smithi072.stdout:(141/142): openssl-1.1.1k-9.el8.x86_64.rpm 3.6 MB/s | 737 kB 00:00 2023-03-24T23:45:00.511 INFO:teuthology.orchestra.run.smithi072.stdout:(142/142): util-linux-2.32.1-41.el8.x86_64.rpm 12 MB/s | 2.6 MB 00:00 2023-03-24T23:45:00.515 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-03-24T23:45:00.516 INFO:teuthology.orchestra.run.smithi072.stdout:Total 15 MB/s | 204 MB 00:13 2023-03-24T23:45:00.520 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2023-03-24T23:45:00.524 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2023-03-24T23:45:01.433 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-03-24T23:45:01.699 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-03-24T23:45:01.700 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-03-24T23:45:03.279 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-03-24T23:45:03.283 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-03-24T23:45:05.816 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2023-03-24T23:45:05.995 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: openssl-libs-1:1.1.1k-9.el8.x86_64 1/1 2023-03-24T23:45:06.267 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : openssl-libs-1:1.1.1k-9.el8.x86_64 1/157 2023-03-24T23:45:06.482 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: openssl-libs-1:1.1.1k-9.el8.x86_64 1/157 2023-03-24T23:45:06.661 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/157 2023-03-24T23:45:06.797 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : libuuid-2.32.1-41.el8.x86_64 3/157 2023-03-24T23:45:06.895 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libuuid-2.32.1-41.el8.x86_64 3/157 2023-03-24T23:45:07.056 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : libblkid-2.32.1-41.el8.x86_64 4/157 2023-03-24T23:45:07.112 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libblkid-2.32.1-41.el8.x86_64 4/157 2023-03-24T23:45:07.311 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/157 2023-03-24T23:45:07.497 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/157 2023-03-24T23:45:07.570 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/157 2023-03-24T23:45:07.767 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 7/157 2023-03-24T23:45:07.931 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/157 2023-03-24T23:45:08.335 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/157 2023-03-24T23:45:08.572 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : librados2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 9/157 2023-03-24T23:45:08.971 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librados2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 9/157 2023-03-24T23:45:09.117 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-03-24T23:45:09.164 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : librbd1-2:18.0.0-3124.gc19e8f2a.el8.x86_64 10/157 2023-03-24T23:45:09.293 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librbd1-2:18.0.0-3124.gc19e8f2a.el8.x86_64 10/157 2023-03-24T23:45:09.313 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: openssl-libs-1:1.1.1k-9.el8.x86_64 1/1 2023-03-24T23:45:09.480 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libcephfs2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 11/157 2023-03-24T23:45:09.565 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : openssl-libs-1:1.1.1k-9.el8.x86_64 1/157 2023-03-24T23:45:09.599 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libcephfs2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 11/157 2023-03-24T23:45:09.779 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: openssl-libs-1:1.1.1k-9.el8.x86_64 1/157 2023-03-24T23:45:09.828 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rados-2:18.0.0-3124.gc19e8f2a.el8.x86_64 12/157 2023-03-24T23:45:09.980 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/157 2023-03-24T23:45:10.022 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libpq-13.5-1.el8.x86_64 13/157 2023-03-24T23:45:10.250 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : elfutils-libelf-0.188-3.el8.x86_64 14/157 2023-03-24T23:45:10.492 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 15/157 2023-03-24T23:45:10.550 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : libuuid-2.32.1-41.el8.x86_64 3/157 2023-03-24T23:45:10.803 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libuuid-2.32.1-41.el8.x86_64 3/157 2023-03-24T23:45:10.967 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : libblkid-2.32.1-41.el8.x86_64 4/157 2023-03-24T23:45:10.976 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/157 2023-03-24T23:45:11.039 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libblkid-2.32.1-41.el8.x86_64 4/157 2023-03-24T23:45:11.240 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/157 2023-03-24T23:45:11.301 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : postgresql-10.23-1.module_el8+300+2c53004c.x86_6 17/157 2023-03-24T23:45:11.391 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/157 2023-03-24T23:45:11.459 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/157 2023-03-24T23:45:11.480 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libcephsqlite-2:18.0.0-3124.gc19e8f2a.el8.x86_64 18/157 2023-03-24T23:45:11.593 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libcephsqlite-2:18.0.0-3124.gc19e8f2a.el8.x86_64 18/157 2023-03-24T23:45:11.685 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 7/157 2023-03-24T23:45:11.763 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libradosstriper1-2:18.0.0-3124.gc19e8f2a.el8.x86 19/157 2023-03-24T23:45:11.876 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libradosstriper1-2:18.0.0-3124.gc19e8f2a.el8.x86 19/157 2023-03-24T23:45:11.916 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/157 2023-03-24T23:45:12.129 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 20/157 2023-03-24T23:45:12.276 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : zlib-devel-1.2.11-19.el8.x86_64 21/157 2023-03-24T23:45:12.319 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/157 2023-03-24T23:45:12.430 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libquadmath-8.5.0-18.el8.x86_64 22/157 2023-03-24T23:45:12.541 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : librados2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 9/157 2023-03-24T23:45:12.562 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libquadmath-8.5.0-18.el8.x86_64 22/157 2023-03-24T23:45:12.755 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libgfortran-8.5.0-18.el8.x86_64 23/157 2023-03-24T23:45:12.929 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librados2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 9/157 2023-03-24T23:45:13.117 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : librbd1-2:18.0.0-3124.gc19e8f2a.el8.x86_64 10/157 2023-03-24T23:45:13.247 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librbd1-2:18.0.0-3124.gc19e8f2a.el8.x86_64 10/157 2023-03-24T23:45:13.346 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libgfortran-8.5.0-18.el8.x86_64 23/157 2023-03-24T23:45:13.408 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libcephfs2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 11/157 2023-03-24T23:45:13.504 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libcephfs2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 11/157 2023-03-24T23:45:13.538 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : openblas-0.3.15-4.el8.x86_64 24/157 2023-03-24T23:45:13.713 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rados-2:18.0.0-3124.gc19e8f2a.el8.x86_64 12/157 2023-03-24T23:45:13.947 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libpq-13.5-1.el8.x86_64 13/157 2023-03-24T23:45:14.158 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 24/157 2023-03-24T23:45:14.199 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : elfutils-libelf-0.188-3.el8.x86_64 14/157 2023-03-24T23:45:14.296 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 25/157 2023-03-24T23:45:14.452 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 15/157 2023-03-24T23:45:14.861 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/157 2023-03-24T23:45:14.983 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 25/157 2023-03-24T23:45:15.106 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : postgresql-10.23-1.module_el8+300+2c53004c.x86_6 17/157 2023-03-24T23:45:15.249 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libcephsqlite-2:18.0.0-3124.gc19e8f2a.el8.x86_64 18/157 2023-03-24T23:45:15.356 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libcephsqlite-2:18.0.0-3124.gc19e8f2a.el8.x86_64 18/157 2023-03-24T23:45:15.372 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 26/157 2023-03-24T23:45:15.524 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libradosstriper1-2:18.0.0-3124.gc19e8f2a.el8.x86 19/157 2023-03-24T23:45:15.594 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 27/157 2023-03-24T23:45:15.641 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libradosstriper1-2:18.0.0-3124.gc19e8f2a.el8.x86 19/157 2023-03-24T23:45:15.824 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 28/157 2023-03-24T23:45:16.041 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 29/157 2023-03-24T23:45:16.180 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 20/157 2023-03-24T23:45:16.219 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-ceph-common-2:18.0.0-3124.gc19e8f2a.el8. 30/157 2023-03-24T23:45:16.325 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : zlib-devel-1.2.11-19.el8.x86_64 21/157 2023-03-24T23:45:16.457 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-ceph-argparse-2:18.0.0-3124.gc19e8f2a.el 31/157 2023-03-24T23:45:16.482 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libquadmath-8.5.0-18.el8.x86_64 22/157 2023-03-24T23:45:16.623 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libquadmath-8.5.0-18.el8.x86_64 22/157 2023-03-24T23:45:16.791 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libgfortran-8.5.0-18.el8.x86_64 23/157 2023-03-24T23:45:16.808 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cephfs-2:18.0.0-3124.gc19e8f2a.el8.x86_6 32/157 2023-03-24T23:45:17.390 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libgfortran-8.5.0-18.el8.x86_64 23/157 2023-03-24T23:45:17.566 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : openblas-0.3.15-4.el8.x86_64 24/157 2023-03-24T23:45:17.815 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 33/157 2023-03-24T23:45:18.220 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 24/157 2023-03-24T23:45:18.225 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 34/157 2023-03-24T23:45:18.349 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 25/157 2023-03-24T23:45:18.541 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 35/157 2023-03-24T23:45:18.829 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/157 2023-03-24T23:45:19.036 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 25/157 2023-03-24T23:45:19.383 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 26/157 2023-03-24T23:45:19.628 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 27/157 2023-03-24T23:45:19.852 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 28/157 2023-03-24T23:45:20.099 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 29/157 2023-03-24T23:45:20.297 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-ceph-common-2:18.0.0-3124.gc19e8f2a.el8. 30/157 2023-03-24T23:45:20.454 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-ceph-argparse-2:18.0.0-3124.gc19e8f2a.el 31/157 2023-03-24T23:45:20.597 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/157 2023-03-24T23:45:20.683 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cephfs-2:18.0.0-3124.gc19e8f2a.el8.x86_6 32/157 2023-03-24T23:45:20.957 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 37/157 2023-03-24T23:45:21.138 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 38/157 2023-03-24T23:45:21.376 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rsa-4.8-1.el8.noarch 39/157 2023-03-24T23:45:21.571 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : elfutils-libs-0.188-3.el8.x86_64 40/157 2023-03-24T23:45:21.646 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 33/157 2023-03-24T23:45:21.772 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : elfutils-debuginfod-client-0.188-3.el8.x86_64 41/157 2023-03-24T23:45:22.009 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rbd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 42/157 2023-03-24T23:45:22.044 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 34/157 2023-03-24T23:45:22.184 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librados-devel-2:18.0.0-3124.gc19e8f2a.el8.x86_6 43/157 2023-03-24T23:45:22.353 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : libfdisk-2.32.1-41.el8.x86_64 44/157 2023-03-24T23:45:22.386 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 35/157 2023-03-24T23:45:22.429 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libfdisk-2.32.1-41.el8.x86_64 44/157 2023-03-24T23:45:22.607 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/157 2023-03-24T23:45:22.619 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : libmount-2.32.1-41.el8.x86_64 45/157 2023-03-24T23:45:22.671 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libmount-2.32.1-41.el8.x86_64 45/157 2023-03-24T23:45:22.873 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libuuid-devel-2.32.1-41.el8.x86_64 46/157 2023-03-24T23:45:23.056 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 47/157 2023-03-24T23:45:23.220 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : libsmartcols-2.32.1-41.el8.x86_64 48/157 2023-03-24T23:45:23.672 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libsmartcols-2.32.1-41.el8.x86_64 48/157 2023-03-24T23:45:23.978 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : util-linux-2.32.1-41.el8.x86_64 49/157 2023-03-24T23:45:23.992 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: util-linux-2.32.1-41.el8.x86_64 49/157 2023-03-24T23:45:24.180 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 50/157 2023-03-24T23:45:24.413 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/157 2023-03-24T23:45:24.449 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 51/157 2023-03-24T23:45:24.684 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 52/157 2023-03-24T23:45:24.793 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 37/157 2023-03-24T23:45:24.857 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 53/157 2023-03-24T23:45:25.016 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 38/157 2023-03-24T23:45:25.054 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 54/157 2023-03-24T23:45:25.274 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rsa-4.8-1.el8.noarch 39/157 2023-03-24T23:45:25.326 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 55/157 2023-03-24T23:45:25.493 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : elfutils-libs-0.188-3.el8.x86_64 40/157 2023-03-24T23:45:25.546 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 56/157 2023-03-24T23:45:25.683 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : elfutils-debuginfod-client-0.188-3.el8.x86_64 41/157 2023-03-24T23:45:25.795 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 57/157 2023-03-24T23:45:25.854 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rbd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 42/157 2023-03-24T23:45:26.038 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 58/157 2023-03-24T23:45:26.041 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librados-devel-2:18.0.0-3124.gc19e8f2a.el8.x86_6 43/157 2023-03-24T23:45:26.223 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : libfdisk-2.32.1-41.el8.x86_64 44/157 2023-03-24T23:45:26.224 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 59/157 2023-03-24T23:45:26.302 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libfdisk-2.32.1-41.el8.x86_64 44/157 2023-03-24T23:45:26.405 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 60/157 2023-03-24T23:45:26.522 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : libmount-2.32.1-41.el8.x86_64 45/157 2023-03-24T23:45:26.588 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libmount-2.32.1-41.el8.x86_64 45/157 2023-03-24T23:45:26.599 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 61/157 2023-03-24T23:45:26.769 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libuuid-devel-2.32.1-41.el8.x86_64 46/157 2023-03-24T23:45:26.784 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-portend-2.6-1.el8.noarch 62/157 2023-03-24T23:45:26.982 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 47/157 2023-03-24T23:45:27.034 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 63/157 2023-03-24T23:45:27.148 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : libsmartcols-2.32.1-41.el8.x86_64 48/157 2023-03-24T23:45:27.240 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 64/157 2023-03-24T23:45:27.515 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 65/157 2023-03-24T23:45:27.665 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libsmartcols-2.32.1-41.el8.x86_64 48/157 2023-03-24T23:45:27.777 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 66/157 2023-03-24T23:45:28.007 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : util-linux-2.32.1-41.el8.x86_64 49/157 2023-03-24T23:45:28.022 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: util-linux-2.32.1-41.el8.x86_64 49/157 2023-03-24T23:45:28.131 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 67/157 2023-03-24T23:45:28.243 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 50/157 2023-03-24T23:45:28.436 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 68/157 2023-03-24T23:45:28.513 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 51/157 2023-03-24T23:45:28.635 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 69/157 2023-03-24T23:45:28.729 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 52/157 2023-03-24T23:45:28.910 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 53/157 2023-03-24T23:45:29.109 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 54/157 2023-03-24T23:45:29.397 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 55/157 2023-03-24T23:45:29.473 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 70/157 2023-03-24T23:45:29.642 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 56/157 2023-03-24T23:45:29.685 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librgw2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 71/157 2023-03-24T23:45:29.744 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librgw2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 71/157 2023-03-24T23:45:29.891 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 57/157 2023-03-24T23:45:29.973 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rgw-2:18.0.0-3124.gc19e8f2a.el8.x86_64 72/157 2023-03-24T23:45:30.109 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 58/157 2023-03-24T23:45:30.254 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 73/157 2023-03-24T23:45:30.312 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 59/157 2023-03-24T23:45:30.518 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 60/157 2023-03-24T23:45:30.668 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 74/157 2023-03-24T23:45:30.728 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 61/157 2023-03-24T23:45:30.914 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-portend-2.6-1.el8.noarch 62/157 2023-03-24T23:45:30.988 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 75/157 2023-03-24T23:45:31.147 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 63/157 2023-03-24T23:45:31.245 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 76/157 2023-03-24T23:45:31.412 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 64/157 2023-03-24T23:45:31.506 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 77/157 2023-03-24T23:45:31.754 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 65/157 2023-03-24T23:45:31.765 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 78/157 2023-03-24T23:45:32.023 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 66/157 2023-03-24T23:45:32.385 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 67/157 2023-03-24T23:45:32.681 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 68/157 2023-03-24T23:45:32.848 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 79/157 2023-03-24T23:45:32.913 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 69/157 2023-03-24T23:45:33.283 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 80/157 2023-03-24T23:45:33.592 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 81/157 2023-03-24T23:45:33.783 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 70/157 2023-03-24T23:45:33.875 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 82/157 2023-03-24T23:45:33.988 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librgw2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 71/157 2023-03-24T23:45:34.063 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librgw2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 71/157 2023-03-24T23:45:34.249 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 83/157 2023-03-24T23:45:34.309 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rgw-2:18.0.0-3124.gc19e8f2a.el8.x86_64 72/157 2023-03-24T23:45:34.557 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 73/157 2023-03-24T23:45:34.560 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 84/157 2023-03-24T23:45:34.812 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 85/157 2023-03-24T23:45:35.005 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 74/157 2023-03-24T23:45:35.040 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : pcre2-utf32-10.32-3.el8.x86_64 86/157 2023-03-24T23:45:35.325 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 75/157 2023-03-24T23:45:35.382 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : pcre2-utf16-10.32-3.el8.x86_64 87/157 2023-03-24T23:45:35.582 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 76/157 2023-03-24T23:45:35.595 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : pcre2-devel-10.32-3.el8.x86_64 88/157 2023-03-24T23:45:35.771 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libverto-devel-0.3.2-2.el8.x86_64 89/157 2023-03-24T23:45:35.867 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 77/157 2023-03-24T23:45:36.054 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 90/157 2023-03-24T23:45:36.134 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 78/157 2023-03-24T23:45:36.313 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libselinux-devel-2.9-6.el8.x86_64 91/157 2023-03-24T23:45:36.437 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libcom_err-devel-1.45.6-5.el8.x86_64 92/157 2023-03-24T23:45:36.623 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 93/157 2023-03-24T23:45:36.793 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : krb5-devel-1.18.2-22.el8.x86_64 94/157 2023-03-24T23:45:36.994 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : fuse-2.9.7-16.el8.x86_64 95/157 2023-03-24T23:45:37.232 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 79/157 2023-03-24T23:45:37.244 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 96/157 2023-03-24T23:45:37.492 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : uuid-1.6.2-43.el8.x86_64 97/157 2023-03-24T23:45:37.560 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 97/157 2023-03-24T23:45:37.679 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 80/157 2023-03-24T23:45:37.743 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 98/157 2023-03-24T23:45:37.988 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 81/157 2023-03-24T23:45:38.322 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 82/157 2023-03-24T23:45:38.677 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 83/157 2023-03-24T23:45:38.948 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 84/157 2023-03-24T23:45:39.215 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 85/157 2023-03-24T23:45:39.450 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : pcre2-utf32-10.32-3.el8.x86_64 86/157 2023-03-24T23:45:39.784 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : pcre2-utf16-10.32-3.el8.x86_64 87/157 2023-03-24T23:45:40.007 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : pcre2-devel-10.32-3.el8.x86_64 88/157 2023-03-24T23:45:40.202 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libverto-devel-0.3.2-2.el8.x86_64 89/157 2023-03-24T23:45:40.510 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 90/157 2023-03-24T23:45:40.758 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libselinux-devel-2.9-6.el8.x86_64 91/157 2023-03-24T23:45:40.908 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libcom_err-devel-1.45.6-5.el8.x86_64 92/157 2023-03-24T23:45:41.101 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 93/157 2023-03-24T23:45:41.292 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-common-2:18.0.0-3124.gc19e8f2a.el8.x86_64 99/157 2023-03-24T23:45:41.347 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : krb5-devel-1.18.2-22.el8.x86_64 94/157 2023-03-24T23:45:41.517 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-common-2:18.0.0-3124.gc19e8f2a.el8.x86_64 99/157 2023-03-24T23:45:41.572 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : fuse-2.9.7-16.el8.x86_64 95/157 2023-03-24T23:45:41.795 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 96/157 2023-03-24T23:45:42.111 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-common-2:18.0.0-3124.gc19e8f2a.el8.x86_64 99/157 2023-03-24T23:45:42.453 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-base-2:18.0.0-3124.gc19e8f2a.el8.x86_64 100/157 2023-03-24T23:45:42.459 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : uuid-1.6.2-43.el8.x86_64 97/157 2023-03-24T23:45:42.857 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-base-2:18.0.0-3124.gc19e8f2a.el8.x86_64 100/157 2023-03-24T23:45:42.946 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 97/157 2023-03-24T23:45:43.113 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 98/157 2023-03-24T23:45:44.283 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-selinux-2:18.0.0-3124.gc19e8f2a.el8.x86_64 101/157 2023-03-24T23:45:46.689 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-common-2:18.0.0-3124.gc19e8f2a.el8.x86_64 99/157 2023-03-24T23:45:46.938 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-common-2:18.0.0-3124.gc19e8f2a.el8.x86_64 99/157 2023-03-24T23:45:47.524 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-common-2:18.0.0-3124.gc19e8f2a.el8.x86_64 99/157 2023-03-24T23:45:47.878 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-base-2:18.0.0-3124.gc19e8f2a.el8.x86_64 100/157 2023-03-24T23:45:47.990 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-base-2:18.0.0-3124.gc19e8f2a.el8.x86_64 100/157 2023-03-24T23:45:49.023 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-selinux-2:18.0.0-3124.gc19e8f2a.el8.x86_64 101/157 2023-03-24T23:46:03.498 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-selinux-2:18.0.0-3124.gc19e8f2a.el8.x86_64 101/157 2023-03-24T23:46:03.498 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /sys 2023-03-24T23:46:03.498 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /proc 2023-03-24T23:46:03.499 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /dev 2023-03-24T23:46:03.499 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /run 2023-03-24T23:46:03.499 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /mnt 2023-03-24T23:46:03.499 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /var/tmp 2023-03-24T23:46:03.499 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /home 2023-03-24T23:46:03.499 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /tmp 2023-03-24T23:46:03.499 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /dev 2023-03-24T23:46:03.500 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:46:03.922 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-volume-2:18.0.0-3124.gc19e8f2a.el8.noarch 102/157 2023-03-24T23:46:05.490 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-volume-2:18.0.0-3124.gc19e8f2a.el8.noarch 102/157 2023-03-24T23:46:05.725 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-osd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 103/157 2023-03-24T23:46:05.946 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-osd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 103/157 2023-03-24T23:46:06.155 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mds-2:18.0.0-3124.gc19e8f2a.el8.x86_64 104/157 2023-03-24T23:46:06.607 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mds-2:18.0.0-3124.gc19e8f2a.el8.x86_64 104/157 2023-03-24T23:46:06.813 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mon-2:18.0.0-3124.gc19e8f2a.el8.x86_64 105/157 2023-03-24T23:46:06.838 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mon-2:18.0.0-3124.gc19e8f2a.el8.x86_64 105/157 2023-03-24T23:46:07.095 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 106/157 2023-03-24T23:46:07.323 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 107/157 2023-03-24T23:46:07.479 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 108/157 2023-03-24T23:46:07.554 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 108/157 2023-03-24T23:46:07.823 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : jq-1.6-6.el8.x86_64 109/157 2023-03-24T23:46:08.088 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : automake-1.16.1-7.el8.noarch 110/157 2023-03-24T23:46:09.636 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: cephadm-2:18.0.0-3124.gc19e8f2a.el8.noarch 111/157 2023-03-24T23:46:09.810 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cephadm-2:18.0.0-3124.gc19e8f2a.el8.noarch 111/157 2023-03-24T23:46:09.968 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-3124.gc19e8f2a.e 112/157 2023-03-24T23:46:10.177 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-3124.gc19e8f2a. 113/157 2023-03-24T23:46:10.370 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-3124.gc19e8f2a.el8.noa 114/157 2023-03-24T23:46:10.402 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-selinux-2:18.0.0-3124.gc19e8f2a.el8.x86_64 101/157 2023-03-24T23:46:10.402 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /sys 2023-03-24T23:46:10.402 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /proc 2023-03-24T23:46:10.402 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /dev 2023-03-24T23:46:10.403 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /run 2023-03-24T23:46:10.403 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /mnt 2023-03-24T23:46:10.403 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /var/tmp 2023-03-24T23:46:10.403 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /home 2023-03-24T23:46:10.403 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /tmp 2023-03-24T23:46:10.403 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /dev 2023-03-24T23:46:10.403 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:46:10.588 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-3124.gc19e8f2a.el8.noa 114/157 2023-03-24T23:46:10.818 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-volume-2:18.0.0-3124.gc19e8f2a.el8.noarch 102/157 2023-03-24T23:46:10.931 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-3124.gc19e8f2a.el8.n 115/157 2023-03-24T23:46:11.897 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-3124.gc19e8f2a.el8.n 115/157 2023-03-24T23:46:12.128 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-3124.gc19 116/157 2023-03-24T23:46:12.222 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-3124.gc19 116/157 2023-03-24T23:46:12.381 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-volume-2:18.0.0-3124.gc19e8f2a.el8.noarch 102/157 2023-03-24T23:46:12.625 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-osd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 103/157 2023-03-24T23:46:12.674 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-3124.gc19e8f2a.el 117/157 2023-03-24T23:46:12.843 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-osd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 103/157 2023-03-24T23:46:12.907 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-2:18.0.0-3124.gc19e8f2a.el8.x86_64 118/157 2023-03-24T23:46:12.942 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-2:18.0.0-3124.gc19e8f2a.el8.x86_64 118/157 2023-03-24T23:46:13.051 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mds-2:18.0.0-3124.gc19e8f2a.el8.x86_64 104/157 2023-03-24T23:46:13.137 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-rook-2:18.0.0-3124.gc19e8f2a.el8.noarch 119/157 2023-03-24T23:46:13.148 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-3124.gc19e8f2a.el8.noarch 119/157 2023-03-24T23:46:13.343 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-2:18.0.0-3124.gc19e8f2a.el8.x86_64 120/157 2023-03-24T23:46:13.501 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mds-2:18.0.0-3124.gc19e8f2a.el8.x86_64 104/157 2023-03-24T23:46:13.511 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libtool-2.4.6-25.el8.x86_64 121/157 2023-03-24T23:46:13.676 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mon-2:18.0.0-3124.gc19e8f2a.el8.x86_64 105/157 2023-03-24T23:46:13.699 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mon-2:18.0.0-3124.gc19e8f2a.el8.x86_64 105/157 2023-03-24T23:46:13.991 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 106/157 2023-03-24T23:46:14.228 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 107/157 2023-03-24T23:46:14.400 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 108/157 2023-03-24T23:46:14.474 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 108/157 2023-03-24T23:46:14.950 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : jq-1.6-6.el8.x86_64 109/157 2023-03-24T23:46:15.235 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : automake-1.16.1-7.el8.noarch 110/157 2023-03-24T23:46:16.765 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: cephadm-2:18.0.0-3124.gc19e8f2a.el8.noarch 111/157 2023-03-24T23:46:16.960 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : cephadm-2:18.0.0-3124.gc19e8f2a.el8.noarch 111/157 2023-03-24T23:46:17.114 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-3124.gc19e8f2a.e 112/157 2023-03-24T23:46:17.321 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-3124.gc19e8f2a. 113/157 2023-03-24T23:46:17.411 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 121/157 2023-03-24T23:46:17.550 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-3124.gc19e8f2a.el8.noa 114/157 2023-03-24T23:46:17.764 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-3124.gc19e8f2a.el8.noa 114/157 2023-03-24T23:46:18.082 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-3124.gc19e8f2a.el8.n 115/157 2023-03-24T23:46:18.847 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-test-2:18.0.0-3124.gc19e8f2a.el8.x86_64 122/157 2023-03-24T23:46:19.044 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-3124.gc19e8f2a.el8.n 115/157 2023-03-24T23:46:19.089 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-radosgw-2:18.0.0-3124.gc19e8f2a.el8.x86_64 123/157 2023-03-24T23:46:19.123 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-3124.gc19e8f2a.el8.x86_64 123/157 2023-03-24T23:46:19.250 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-3124.gc19 116/157 2023-03-24T23:46:19.340 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-3124.gc19 116/157 2023-03-24T23:46:19.386 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-3124.gc19e8 124/157 2023-03-24T23:46:19.429 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-3124.gc19e8 124/157 2023-03-24T23:46:19.636 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cephfs-mirror-2:18.0.0-3124.gc19e8f2a.el8.x86_64 125/157 2023-03-24T23:46:19.783 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-3124.gc19e8f2a.el 117/157 2023-03-24T23:46:19.984 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-3124.gc19e8f2a.el8.x86_64 125/157 2023-03-24T23:46:19.988 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-2:18.0.0-3124.gc19e8f2a.el8.x86_64 118/157 2023-03-24T23:46:20.024 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-2:18.0.0-3124.gc19e8f2a.el8.x86_64 118/157 2023-03-24T23:46:20.208 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-rook-2:18.0.0-3124.gc19e8f2a.el8.noarch 119/157 2023-03-24T23:46:20.218 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-3124.gc19e8f2a.el8.noarch 119/157 2023-03-24T23:46:20.244 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : rbd-mirror-2:18.0.0-3124.gc19e8f2a.el8.x86_64 126/157 2023-03-24T23:46:20.403 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-2:18.0.0-3124.gc19e8f2a.el8.x86_64 120/157 2023-03-24T23:46:20.453 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: rbd-mirror-2:18.0.0-3124.gc19e8f2a.el8.x86_64 126/157 2023-03-24T23:46:20.608 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libtool-2.4.6-25.el8.x86_64 121/157 2023-03-24T23:46:20.834 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : postgresql-contrib-10.23-1.module_el8+300+2c5300 127/157 2023-03-24T23:46:21.759 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-fuse-2:18.0.0-3124.gc19e8f2a.el8.x86_64 128/157 2023-03-24T23:46:22.058 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : openssl-devel-1:1.1.1k-9.el8.x86_64 129/157 2023-03-24T23:46:22.296 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : util-linux-user-2.32.1-41.el8.x86_64 130/157 2023-03-24T23:46:22.456 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 131/157 2023-03-24T23:46:22.656 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libcephfs-devel-2:18.0.0-3124.gc19e8f2a.el8.x86_ 132/157 2023-03-24T23:46:22.845 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : elfutils-libelf-devel-0.188-3.el8.x86_64 133/157 2023-03-24T23:46:24.501 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 121/157 2023-03-24T23:46:24.898 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: postgresql-server-10.23-1.module_el8+300+2c53004 134/157 2023-03-24T23:46:25.186 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : postgresql-server-10.23-1.module_el8+300+2c53004 134/157 2023-03-24T23:46:25.203 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: postgresql-server-10.23-1.module_el8+300+2c53004 134/157 2023-03-24T23:46:25.527 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cephfs-top-2:18.0.0-3124.gc19e8f2a.el8.noarch 135/157 2023-03-24T23:46:25.734 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : rbd-fuse-2:18.0.0-3124.gc19e8f2a.el8.x86_64 136/157 2023-03-24T23:46:25.942 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-test-2:18.0.0-3124.gc19e8f2a.el8.x86_64 122/157 2023-03-24T23:46:26.038 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : rbd-nbd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 137/157 2023-03-24T23:46:26.163 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-radosgw-2:18.0.0-3124.gc19e8f2a.el8.x86_64 123/157 2023-03-24T23:46:26.196 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-3124.gc19e8f2a.el8.x86_64 123/157 2023-03-24T23:46:26.378 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : openssl-1:1.1.1k-9.el8.x86_64 138/157 2023-03-24T23:46:26.408 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-3124.gc19e8 124/157 2023-03-24T23:46:26.449 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-3124.gc19e8 124/157 2023-03-24T23:46:26.526 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : NetworkManager-initscripts-updown-1:1.40.16-1.el 139/157 2023-03-24T23:46:26.544 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-1.el 139/157 2023-03-24T23:46:26.641 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : cephfs-mirror-2:18.0.0-3124.gc19e8f2a.el8.x86_64 125/157 2023-03-24T23:46:26.732 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 140/157 2023-03-24T23:46:26.935 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : flex-2.6.1-9.el8.x86_64 141/157 2023-03-24T23:46:26.986 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-3124.gc19e8f2a.el8.x86_64 125/157 2023-03-24T23:46:27.017 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 141/157 2023-03-24T23:46:27.216 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : rbd-mirror-2:18.0.0-3124.gc19e8f2a.el8.x86_64 126/157 2023-03-24T23:46:27.243 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : bison-3.0.4-10.el8.x86_64 142/157 2023-03-24T23:46:27.257 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 142/157 2023-03-24T23:46:27.426 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: rbd-mirror-2:18.0.0-3124.gc19e8f2a.el8.x86_64 126/157 2023-03-24T23:46:27.486 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : openssl-1:1.1.1k-7.el8.x86_64 143/157 2023-03-24T23:46:27.636 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : elfutils-debuginfod-client-0.187-4.el8.x86_64 144/157 2023-03-24T23:46:27.789 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : postgresql-contrib-10.23-1.module_el8+300+2c5300 127/157 2023-03-24T23:46:27.798 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : elfutils-libs-0.187-4.el8.x86_64 145/157 2023-03-24T23:46:27.798 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 146/157 2023-03-24T23:46:28.028 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 146/157 2023-03-24T23:46:28.032 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.39.12-1.el 147/157 2023-03-24T23:46:28.146 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : NetworkManager-initscripts-updown-1:1.39.12-1.el 147/157 2023-03-24T23:46:28.146 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 148/157 2023-03-24T23:46:28.370 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 148/157 2023-03-24T23:46:28.543 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : util-linux-user-2.32.1-38.el8.x86_64 149/157 2023-03-24T23:46:28.665 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-fuse-2:18.0.0-3124.gc19e8f2a.el8.x86_64 128/157 2023-03-24T23:46:28.761 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : util-linux-2.32.1-38.el8.x86_64 150/157 2023-03-24T23:46:28.761 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : libfdisk-2.32.1-38.el8.x86_64 151/157 2023-03-24T23:46:28.961 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libfdisk-2.32.1-38.el8.x86_64 151/157 2023-03-24T23:46:28.962 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : libmount-2.32.1-38.el8.x86_64 152/157 2023-03-24T23:46:28.988 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : openssl-devel-1:1.1.1k-9.el8.x86_64 129/157 2023-03-24T23:46:29.161 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libmount-2.32.1-38.el8.x86_64 152/157 2023-03-24T23:46:29.162 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : libblkid-2.32.1-38.el8.x86_64 153/157 2023-03-24T23:46:29.209 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : util-linux-user-2.32.1-41.el8.x86_64 130/157 2023-03-24T23:46:29.378 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libblkid-2.32.1-38.el8.x86_64 153/157 2023-03-24T23:46:29.378 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : libuuid-2.32.1-38.el8.x86_64 154/157 2023-03-24T23:46:29.386 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 131/157 2023-03-24T23:46:29.586 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libcephfs-devel-2:18.0.0-3124.gc19e8f2a.el8.x86_ 132/157 2023-03-24T23:46:29.594 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libuuid-2.32.1-38.el8.x86_64 154/157 2023-03-24T23:46:29.595 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : libsmartcols-2.32.1-38.el8.x86_64 155/157 2023-03-24T23:46:29.778 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libsmartcols-2.32.1-38.el8.x86_64 155/157 2023-03-24T23:46:29.791 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : elfutils-libelf-devel-0.188-3.el8.x86_64 133/157 2023-03-24T23:46:29.953 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : elfutils-libelf-0.187-4.el8.x86_64 156/157 2023-03-24T23:46:29.954 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : openssl-libs-1:1.1.1k-7.el8.x86_64 157/157 2023-03-24T23:46:30.858 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: openssl-libs-1:1.1.1k-7.el8.x86_64 157/157 2023-03-24T23:46:30.859 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-2:18.0.0-3124.gc19e8f2a.el8.x86_64 1/157 2023-03-24T23:46:30.859 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-base-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2/157 2023-03-24T23:46:30.859 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-common-2:18.0.0-3124.gc19e8f2a.el8.x86_64 3/157 2023-03-24T23:46:30.859 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-fuse-2:18.0.0-3124.gc19e8f2a.el8.x86_64 4/157 2023-03-24T23:46:30.859 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-3124.gc19e8 5/157 2023-03-24T23:46:30.859 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mds-2:18.0.0-3124.gc19e8f2a.el8.x86_64 6/157 2023-03-24T23:46:30.859 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-2:18.0.0-3124.gc19e8f2a.el8.x86_64 7/157 2023-03-24T23:46:30.860 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mon-2:18.0.0-3124.gc19e8f2a.el8.x86_64 8/157 2023-03-24T23:46:30.860 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-osd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 9/157 2023-03-24T23:46:30.860 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-radosgw-2:18.0.0-3124.gc19e8f2a.el8.x86_64 10/157 2023-03-24T23:46:30.860 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-selinux-2:18.0.0-3124.gc19e8f2a.el8.x86_64 11/157 2023-03-24T23:46:30.860 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-test-2:18.0.0-3124.gc19e8f2a.el8.x86_64 12/157 2023-03-24T23:46:30.860 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cephfs-mirror-2:18.0.0-3124.gc19e8f2a.el8.x86_64 13/157 2023-03-24T23:46:30.860 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libcephfs-devel-2:18.0.0-3124.gc19e8f2a.el8.x86_ 14/157 2023-03-24T23:46:30.861 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libcephfs2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 15/157 2023-03-24T23:46:30.861 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libcephsqlite-2:18.0.0-3124.gc19e8f2a.el8.x86_64 16/157 2023-03-24T23:46:30.861 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librados-devel-2:18.0.0-3124.gc19e8f2a.el8.x86_6 17/157 2023-03-24T23:46:30.861 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libradosstriper1-2:18.0.0-3124.gc19e8f2a.el8.x86 18/157 2023-03-24T23:46:30.861 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librgw2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 19/157 2023-03-24T23:46:30.862 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-ceph-argparse-2:18.0.0-3124.gc19e8f2a.el 20/157 2023-03-24T23:46:30.862 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-ceph-common-2:18.0.0-3124.gc19e8f2a.el8. 21/157 2023-03-24T23:46:30.863 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cephfs-2:18.0.0-3124.gc19e8f2a.el8.x86_6 22/157 2023-03-24T23:46:30.863 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rados-2:18.0.0-3124.gc19e8f2a.el8.x86_64 23/157 2023-03-24T23:46:30.863 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rbd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 24/157 2023-03-24T23:46:30.863 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rgw-2:18.0.0-3124.gc19e8f2a.el8.x86_64 25/157 2023-03-24T23:46:30.863 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : rbd-fuse-2:18.0.0-3124.gc19e8f2a.el8.x86_64 26/157 2023-03-24T23:46:30.863 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : rbd-mirror-2:18.0.0-3124.gc19e8f2a.el8.x86_64 27/157 2023-03-24T23:46:30.864 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : rbd-nbd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 28/157 2023-03-24T23:46:30.864 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-3124.gc19e8f2a. 29/157 2023-03-24T23:46:30.864 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-3124.gc19e8f2a.el8.noa 30/157 2023-03-24T23:46:30.864 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-3124.gc19e8f2a.el8.n 31/157 2023-03-24T23:46:30.864 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-3124.gc19 32/157 2023-03-24T23:46:30.864 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-3124.gc19e8f2a.el 33/157 2023-03-24T23:46:30.864 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-rook-2:18.0.0-3124.gc19e8f2a.el8.noarch 34/157 2023-03-24T23:46:30.864 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-3124.gc19e8f2a.e 35/157 2023-03-24T23:46:30.865 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-volume-2:18.0.0-3124.gc19e8f2a.el8.noarch 36/157 2023-03-24T23:46:30.865 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cephadm-2:18.0.0-3124.gc19e8f2a.el8.noarch 37/157 2023-03-24T23:46:30.865 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cephfs-top-2:18.0.0-3124.gc19e8f2a.el8.noarch 38/157 2023-03-24T23:46:30.865 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : automake-1.16.1-7.el8.noarch 39/157 2023-03-24T23:46:30.865 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : bison-3.0.4-10.el8.x86_64 40/157 2023-03-24T23:46:30.865 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : flex-2.6.1-9.el8.x86_64 41/157 2023-03-24T23:46:30.865 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : jq-1.6-6.el8.x86_64 42/157 2023-03-24T23:46:30.866 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libpq-13.5-1.el8.x86_64 43/157 2023-03-24T23:46:30.866 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 44/157 2023-03-24T23:46:30.866 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 45/157 2023-03-24T23:46:30.866 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 46/157 2023-03-24T23:46:30.866 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 47/157 2023-03-24T23:46:30.866 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 48/157 2023-03-24T23:46:30.866 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 49/157 2023-03-24T23:46:30.866 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : postgresql-10.23-1.module_el8+300+2c53004c.x86_6 50/157 2023-03-24T23:46:30.867 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : postgresql-contrib-10.23-1.module_el8+300+2c5300 51/157 2023-03-24T23:46:30.867 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : postgresql-server-10.23-1.module_el8+300+2c53004 52/157 2023-03-24T23:46:30.867 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 53/157 2023-03-24T23:46:30.867 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 54/157 2023-03-24T23:46:30.867 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 55/157 2023-03-24T23:46:30.867 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 56/157 2023-03-24T23:46:30.867 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 57/157 2023-03-24T23:46:30.868 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 58/157 2023-03-24T23:46:30.868 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 59/157 2023-03-24T23:46:30.868 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 60/157 2023-03-24T23:46:30.868 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 61/157 2023-03-24T23:46:30.868 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 62/157 2023-03-24T23:46:30.868 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 63/157 2023-03-24T23:46:30.868 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 64/157 2023-03-24T23:46:30.868 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 65/157 2023-03-24T23:46:30.869 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 66/157 2023-03-24T23:46:30.869 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 67/157 2023-03-24T23:46:30.869 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 68/157 2023-03-24T23:46:30.869 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 69/157 2023-03-24T23:46:30.869 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 70/157 2023-03-24T23:46:30.869 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : elfutils-libelf-devel-0.188-3.el8.x86_64 71/157 2023-03-24T23:46:30.869 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : fuse-2.9.7-16.el8.x86_64 72/157 2023-03-24T23:46:30.869 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 73/157 2023-03-24T23:46:30.870 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : krb5-devel-1.18.2-22.el8.x86_64 74/157 2023-03-24T23:46:30.870 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 75/157 2023-03-24T23:46:30.870 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libcom_err-devel-1.45.6-5.el8.x86_64 76/157 2023-03-24T23:46:30.870 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libgfortran-8.5.0-18.el8.x86_64 77/157 2023-03-24T23:46:30.870 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libquadmath-8.5.0-18.el8.x86_64 78/157 2023-03-24T23:46:30.871 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 79/157 2023-03-24T23:46:30.871 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libselinux-devel-2.9-6.el8.x86_64 80/157 2023-03-24T23:46:30.871 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 81/157 2023-03-24T23:46:30.871 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libuuid-devel-2.32.1-41.el8.x86_64 82/157 2023-03-24T23:46:30.871 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libverto-devel-0.3.2-2.el8.x86_64 83/157 2023-03-24T23:46:30.871 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openssl-devel-1:1.1.1k-9.el8.x86_64 84/157 2023-03-24T23:46:30.871 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : pcre2-devel-10.32-3.el8.x86_64 85/157 2023-03-24T23:46:30.871 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : pcre2-utf16-10.32-3.el8.x86_64 86/157 2023-03-24T23:46:30.872 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : pcre2-utf32-10.32-3.el8.x86_64 87/157 2023-03-24T23:46:30.872 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 88/157 2023-03-24T23:46:30.872 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 89/157 2023-03-24T23:46:30.872 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 90/157 2023-03-24T23:46:30.872 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 91/157 2023-03-24T23:46:30.872 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 92/157 2023-03-24T23:46:30.872 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 93/157 2023-03-24T23:46:30.873 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 94/157 2023-03-24T23:46:30.873 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : zlib-devel-1.2.11-19.el8.x86_64 95/157 2023-03-24T23:46:30.873 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 96/157 2023-03-24T23:46:30.873 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 97/157 2023-03-24T23:46:30.873 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 98/157 2023-03-24T23:46:30.873 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 99/157 2023-03-24T23:46:30.873 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 100/157 2023-03-24T23:46:30.874 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 101/157 2023-03-24T23:46:30.874 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 102/157 2023-03-24T23:46:30.874 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 103/157 2023-03-24T23:46:30.874 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 104/157 2023-03-24T23:46:30.874 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 105/157 2023-03-24T23:46:30.874 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 106/157 2023-03-24T23:46:30.874 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 107/157 2023-03-24T23:46:30.874 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 108/157 2023-03-24T23:46:30.875 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 109/157 2023-03-24T23:46:30.875 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 110/157 2023-03-24T23:46:30.875 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 111/157 2023-03-24T23:46:30.875 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 112/157 2023-03-24T23:46:30.875 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 113/157 2023-03-24T23:46:30.875 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-portend-2.6-1.el8.noarch 114/157 2023-03-24T23:46:30.875 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 115/157 2023-03-24T23:46:30.875 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 116/157 2023-03-24T23:46:30.876 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 117/157 2023-03-24T23:46:30.876 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 118/157 2023-03-24T23:46:30.876 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 119/157 2023-03-24T23:46:30.876 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 120/157 2023-03-24T23:46:30.876 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 121/157 2023-03-24T23:46:30.876 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 122/157 2023-03-24T23:46:30.876 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 123/157 2023-03-24T23:46:30.876 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 124/157 2023-03-24T23:46:30.877 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 125/157 2023-03-24T23:46:30.877 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 126/157 2023-03-24T23:46:30.877 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 127/157 2023-03-24T23:46:30.877 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librados2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 128/157 2023-03-24T23:46:30.877 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 129/157 2023-03-24T23:46:30.877 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librbd1-2:18.0.0-3124.gc19e8f2a.el8.x86_64 130/157 2023-03-24T23:46:30.877 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 131/157 2023-03-24T23:46:30.878 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-1.el 132/157 2023-03-24T23:46:30.878 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : NetworkManager-initscripts-updown-1:1.39.12-1.el 133/157 2023-03-24T23:46:30.878 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : elfutils-debuginfod-client-0.188-3.el8.x86_64 134/157 2023-03-24T23:46:30.878 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : elfutils-debuginfod-client-0.187-4.el8.x86_64 135/157 2023-03-24T23:46:30.878 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : elfutils-libelf-0.188-3.el8.x86_64 136/157 2023-03-24T23:46:30.878 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : elfutils-libelf-0.187-4.el8.x86_64 137/157 2023-03-24T23:46:30.878 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : elfutils-libs-0.188-3.el8.x86_64 138/157 2023-03-24T23:46:30.878 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : elfutils-libs-0.187-4.el8.x86_64 139/157 2023-03-24T23:46:30.879 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libblkid-2.32.1-41.el8.x86_64 140/157 2023-03-24T23:46:30.879 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libblkid-2.32.1-38.el8.x86_64 141/157 2023-03-24T23:46:30.879 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libfdisk-2.32.1-41.el8.x86_64 142/157 2023-03-24T23:46:30.879 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libfdisk-2.32.1-38.el8.x86_64 143/157 2023-03-24T23:46:30.879 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libmount-2.32.1-41.el8.x86_64 144/157 2023-03-24T23:46:30.879 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libmount-2.32.1-38.el8.x86_64 145/157 2023-03-24T23:46:30.879 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libsmartcols-2.32.1-41.el8.x86_64 146/157 2023-03-24T23:46:30.879 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libsmartcols-2.32.1-38.el8.x86_64 147/157 2023-03-24T23:46:30.880 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libuuid-2.32.1-41.el8.x86_64 148/157 2023-03-24T23:46:30.880 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libuuid-2.32.1-38.el8.x86_64 149/157 2023-03-24T23:46:30.880 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openssl-1:1.1.1k-9.el8.x86_64 150/157 2023-03-24T23:46:30.880 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openssl-1:1.1.1k-7.el8.x86_64 151/157 2023-03-24T23:46:30.880 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openssl-libs-1:1.1.1k-9.el8.x86_64 152/157 2023-03-24T23:46:30.880 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openssl-libs-1:1.1.1k-7.el8.x86_64 153/157 2023-03-24T23:46:30.880 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : util-linux-2.32.1-41.el8.x86_64 154/157 2023-03-24T23:46:30.881 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : util-linux-2.32.1-38.el8.x86_64 155/157 2023-03-24T23:46:30.881 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : util-linux-user-2.32.1-41.el8.x86_64 156/157 2023-03-24T23:46:31.859 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: postgresql-server-10.23-1.module_el8+300+2c53004 134/157 2023-03-24T23:46:32.157 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : postgresql-server-10.23-1.module_el8+300+2c53004 134/157 2023-03-24T23:46:32.175 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: postgresql-server-10.23-1.module_el8+300+2c53004 134/157 2023-03-24T23:46:32.507 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : cephfs-top-2:18.0.0-3124.gc19e8f2a.el8.noarch 135/157 2023-03-24T23:46:32.722 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : rbd-fuse-2:18.0.0-3124.gc19e8f2a.el8.x86_64 136/157 2023-03-24T23:46:32.920 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : util-linux-user-2.32.1-38.el8.x86_64 157/157 2023-03-24T23:46:32.920 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:46:32.920 INFO:teuthology.orchestra.run.smithi146.stdout:Upgraded: 2023-03-24T23:46:32.920 INFO:teuthology.orchestra.run.smithi146.stdout: NetworkManager-initscripts-updown-1:1.40.16-1.el8.noarch 2023-03-24T23:46:32.920 INFO:teuthology.orchestra.run.smithi146.stdout: elfutils-debuginfod-client-0.188-3.el8.x86_64 2023-03-24T23:46:32.920 INFO:teuthology.orchestra.run.smithi146.stdout: elfutils-libelf-0.188-3.el8.x86_64 2023-03-24T23:46:32.921 INFO:teuthology.orchestra.run.smithi146.stdout: elfutils-libs-0.188-3.el8.x86_64 2023-03-24T23:46:32.921 INFO:teuthology.orchestra.run.smithi146.stdout: libblkid-2.32.1-41.el8.x86_64 2023-03-24T23:46:32.921 INFO:teuthology.orchestra.run.smithi146.stdout: libfdisk-2.32.1-41.el8.x86_64 2023-03-24T23:46:32.921 INFO:teuthology.orchestra.run.smithi146.stdout: libmount-2.32.1-41.el8.x86_64 2023-03-24T23:46:32.921 INFO:teuthology.orchestra.run.smithi146.stdout: librados2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:32.921 INFO:teuthology.orchestra.run.smithi146.stdout: librbd1-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:32.921 INFO:teuthology.orchestra.run.smithi146.stdout: libsmartcols-2.32.1-41.el8.x86_64 2023-03-24T23:46:32.922 INFO:teuthology.orchestra.run.smithi146.stdout: libuuid-2.32.1-41.el8.x86_64 2023-03-24T23:46:32.922 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-1:1.1.1k-9.el8.x86_64 2023-03-24T23:46:32.922 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-libs-1:1.1.1k-9.el8.x86_64 2023-03-24T23:46:32.922 INFO:teuthology.orchestra.run.smithi146.stdout: util-linux-2.32.1-41.el8.x86_64 2023-03-24T23:46:32.922 INFO:teuthology.orchestra.run.smithi146.stdout: util-linux-user-2.32.1-41.el8.x86_64 2023-03-24T23:46:32.922 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2023-03-24T23:46:32.922 INFO:teuthology.orchestra.run.smithi146.stdout: automake-1.16.1-7.el8.noarch 2023-03-24T23:46:32.923 INFO:teuthology.orchestra.run.smithi146.stdout: bison-3.0.4-10.el8.x86_64 2023-03-24T23:46:32.923 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:32.923 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-base-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:32.923 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-common-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:32.923 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-fuse-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:32.923 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-grafana-dashboards-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T23:46:32.923 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-immutable-object-cache-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:32.923 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mds-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:32.924 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:32.924 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-cephadm-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T23:46:32.924 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-dashboard-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T23:46:32.924 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-diskprediction-local-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T23:46:32.924 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-modules-core-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T23:46:32.924 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-rook-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T23:46:32.924 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mon-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:32.924 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-osd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:32.925 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-prometheus-alerts-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T23:46:32.925 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-radosgw-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:32.925 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-selinux-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:32.925 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-test-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:32.925 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-volume-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T23:46:32.925 INFO:teuthology.orchestra.run.smithi146.stdout: cephadm-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T23:46:32.925 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-mirror-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:32.926 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-top-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T23:46:32.926 INFO:teuthology.orchestra.run.smithi146.stdout: elfutils-libelf-devel-0.188-3.el8.x86_64 2023-03-24T23:46:32.926 INFO:teuthology.orchestra.run.smithi146.stdout: flex-2.6.1-9.el8.x86_64 2023-03-24T23:46:32.926 INFO:teuthology.orchestra.run.smithi146.stdout: fuse-2.9.7-16.el8.x86_64 2023-03-24T23:46:32.926 INFO:teuthology.orchestra.run.smithi146.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-03-24T23:46:32.926 INFO:teuthology.orchestra.run.smithi146.stdout: jq-1.6-6.el8.x86_64 2023-03-24T23:46:32.926 INFO:teuthology.orchestra.run.smithi146.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-03-24T23:46:32.926 INFO:teuthology.orchestra.run.smithi146.stdout: krb5-devel-1.18.2-22.el8.x86_64 2023-03-24T23:46:32.927 INFO:teuthology.orchestra.run.smithi146.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-03-24T23:46:32.927 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs-devel-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:32.927 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:32.927 INFO:teuthology.orchestra.run.smithi146.stdout: libcephsqlite-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:32.927 INFO:teuthology.orchestra.run.smithi146.stdout: libcom_err-devel-1.45.6-5.el8.x86_64 2023-03-24T23:46:32.927 INFO:teuthology.orchestra.run.smithi146.stdout: libgfortran-8.5.0-18.el8.x86_64 2023-03-24T23:46:32.927 INFO:teuthology.orchestra.run.smithi146.stdout: liboath-2.6.2-3.el8.x86_64 2023-03-24T23:46:32.927 INFO:teuthology.orchestra.run.smithi146.stdout: libpq-13.5-1.el8.x86_64 2023-03-24T23:46:32.928 INFO:teuthology.orchestra.run.smithi146.stdout: libquadmath-8.5.0-18.el8.x86_64 2023-03-24T23:46:32.928 INFO:teuthology.orchestra.run.smithi146.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-03-24T23:46:32.928 INFO:teuthology.orchestra.run.smithi146.stdout: librados-devel-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:32.928 INFO:teuthology.orchestra.run.smithi146.stdout: libradosstriper1-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:32.928 INFO:teuthology.orchestra.run.smithi146.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-03-24T23:46:32.928 INFO:teuthology.orchestra.run.smithi146.stdout: librgw2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:32.928 INFO:teuthology.orchestra.run.smithi146.stdout: libselinux-devel-2.9-6.el8.x86_64 2023-03-24T23:46:32.929 INFO:teuthology.orchestra.run.smithi146.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-03-24T23:46:32.929 INFO:teuthology.orchestra.run.smithi146.stdout: libtool-2.4.6-25.el8.x86_64 2023-03-24T23:46:32.929 INFO:teuthology.orchestra.run.smithi146.stdout: libunwind-1.3.1-3.el8.x86_64 2023-03-24T23:46:32.929 INFO:teuthology.orchestra.run.smithi146.stdout: libuuid-devel-2.32.1-41.el8.x86_64 2023-03-24T23:46:32.929 INFO:teuthology.orchestra.run.smithi146.stdout: libverto-devel-0.3.2-2.el8.x86_64 2023-03-24T23:46:32.929 INFO:teuthology.orchestra.run.smithi146.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-03-24T23:46:32.929 INFO:teuthology.orchestra.run.smithi146.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-03-24T23:46:32.929 INFO:teuthology.orchestra.run.smithi146.stdout: openblas-0.3.15-4.el8.x86_64 2023-03-24T23:46:32.930 INFO:teuthology.orchestra.run.smithi146.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-03-24T23:46:32.930 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-devel-1:1.1.1k-9.el8.x86_64 2023-03-24T23:46:32.930 INFO:teuthology.orchestra.run.smithi146.stdout: pcre2-devel-10.32-3.el8.x86_64 2023-03-24T23:46:32.930 INFO:teuthology.orchestra.run.smithi146.stdout: pcre2-utf16-10.32-3.el8.x86_64 2023-03-24T23:46:32.930 INFO:teuthology.orchestra.run.smithi146.stdout: pcre2-utf32-10.32-3.el8.x86_64 2023-03-24T23:46:32.930 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-10.23-1.module_el8+300+2c53004c.x86_64 2023-03-24T23:46:32.930 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-contrib-10.23-1.module_el8+300+2c53004c.x86_64 2023-03-24T23:46:32.931 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-server-10.23-1.module_el8+300+2c53004c.x86_64 2023-03-24T23:46:32.931 INFO:teuthology.orchestra.run.smithi146.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-03-24T23:46:32.931 INFO:teuthology.orchestra.run.smithi146.stdout: python3-babel-2.5.1-7.el8.noarch 2023-03-24T23:46:32.931 INFO:teuthology.orchestra.run.smithi146.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-03-24T23:46:32.931 INFO:teuthology.orchestra.run.smithi146.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-03-24T23:46:32.931 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-03-24T23:46:32.931 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-argparse-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:32.931 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-common-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:32.932 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cephfs-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:32.932 INFO:teuthology.orchestra.run.smithi146.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-03-24T23:46:32.932 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-03-24T23:46:32.932 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-03-24T23:46:32.932 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-03-24T23:46:32.932 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-03-24T23:46:32.932 INFO:teuthology.orchestra.run.smithi146.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-03-24T23:46:32.932 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-6.2-6.el8.noarch 2023-03-24T23:46:32.933 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-03-24T23:46:32.933 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-03-24T23:46:32.933 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-03-24T23:46:32.933 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-03-24T23:46:32.933 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-03-24T23:46:32.933 INFO:teuthology.orchestra.run.smithi146.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-03-24T23:46:32.933 INFO:teuthology.orchestra.run.smithi146.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-03-24T23:46:32.934 INFO:teuthology.orchestra.run.smithi146.stdout: python3-mako-1.0.6-14.el8.noarch 2023-03-24T23:46:32.934 INFO:teuthology.orchestra.run.smithi146.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-03-24T23:46:32.934 INFO:teuthology.orchestra.run.smithi146.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-03-24T23:46:32.934 INFO:teuthology.orchestra.run.smithi146.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-03-24T23:46:32.934 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-03-24T23:46:32.934 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-03-24T23:46:32.934 INFO:teuthology.orchestra.run.smithi146.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-03-24T23:46:32.934 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-03-24T23:46:32.935 INFO:teuthology.orchestra.run.smithi146.stdout: python3-portend-2.6-1.el8.noarch 2023-03-24T23:46:32.935 INFO:teuthology.orchestra.run.smithi146.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-03-24T23:46:32.935 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-03-24T23:46:32.935 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-03-24T23:46:32.935 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-03-24T23:46:32.935 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pycparser-2.14-14.el8.noarch 2023-03-24T23:46:32.935 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pytz-2017.2-9.el8.noarch 2023-03-24T23:46:32.935 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rados-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:32.936 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rbd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:32.936 INFO:teuthology.orchestra.run.smithi146.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-03-24T23:46:32.936 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-03-24T23:46:32.936 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rgw-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:32.936 INFO:teuthology.orchestra.run.smithi146.stdout: python3-routes-2.4.1-12.el8.noarch 2023-03-24T23:46:32.936 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rsa-4.8-1.el8.noarch 2023-03-24T23:46:32.936 INFO:teuthology.orchestra.run.smithi146.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-03-24T23:46:32.937 INFO:teuthology.orchestra.run.smithi146.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-03-24T23:46:32.937 INFO:teuthology.orchestra.run.smithi146.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-03-24T23:46:32.937 INFO:teuthology.orchestra.run.smithi146.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-03-24T23:46:32.937 INFO:teuthology.orchestra.run.smithi146.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-03-24T23:46:32.937 INFO:teuthology.orchestra.run.smithi146.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-03-24T23:46:32.937 INFO:teuthology.orchestra.run.smithi146.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-03-24T23:46:32.937 INFO:teuthology.orchestra.run.smithi146.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-03-24T23:46:32.937 INFO:teuthology.orchestra.run.smithi146.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-03-24T23:46:32.938 INFO:teuthology.orchestra.run.smithi146.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-03-24T23:46:32.938 INFO:teuthology.orchestra.run.smithi146.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-03-24T23:46:32.938 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-fuse-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:32.938 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-mirror-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:32.938 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-nbd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:32.938 INFO:teuthology.orchestra.run.smithi146.stdout: socat-1.7.4.1-1.el8.x86_64 2023-03-24T23:46:32.938 INFO:teuthology.orchestra.run.smithi146.stdout: thrift-0.13.0-2.el8.x86_64 2023-03-24T23:46:32.938 INFO:teuthology.orchestra.run.smithi146.stdout: uuid-1.6.2-43.el8.x86_64 2023-03-24T23:46:32.939 INFO:teuthology.orchestra.run.smithi146.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-03-24T23:46:32.939 INFO:teuthology.orchestra.run.smithi146.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-03-24T23:46:32.939 INFO:teuthology.orchestra.run.smithi146.stdout: zlib-devel-1.2.11-19.el8.x86_64 2023-03-24T23:46:32.939 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:46:32.940 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2023-03-24T23:46:33.043 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : rbd-nbd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 137/157 2023-03-24T23:46:33.093 DEBUG:teuthology.parallel:result is None 2023-03-24T23:46:33.342 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : openssl-1:1.1.1k-9.el8.x86_64 138/157 2023-03-24T23:46:33.506 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : NetworkManager-initscripts-updown-1:1.40.16-1.el 139/157 2023-03-24T23:46:33.523 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-1.el 139/157 2023-03-24T23:46:33.721 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 140/157 2023-03-24T23:46:33.940 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : flex-2.6.1-9.el8.x86_64 141/157 2023-03-24T23:46:34.022 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 141/157 2023-03-24T23:46:34.248 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : bison-3.0.4-10.el8.x86_64 142/157 2023-03-24T23:46:34.262 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 142/157 2023-03-24T23:46:34.508 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : openssl-1:1.1.1k-7.el8.x86_64 143/157 2023-03-24T23:46:34.682 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : elfutils-debuginfod-client-0.187-4.el8.x86_64 144/157 2023-03-24T23:46:34.863 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : elfutils-libs-0.187-4.el8.x86_64 145/157 2023-03-24T23:46:34.863 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 146/157 2023-03-24T23:46:35.100 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 146/157 2023-03-24T23:46:35.105 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.39.12-1.el 147/157 2023-03-24T23:46:35.209 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : NetworkManager-initscripts-updown-1:1.39.12-1.el 147/157 2023-03-24T23:46:35.209 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 148/157 2023-03-24T23:46:35.442 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 148/157 2023-03-24T23:46:35.614 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : util-linux-user-2.32.1-38.el8.x86_64 149/157 2023-03-24T23:46:35.883 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : util-linux-2.32.1-38.el8.x86_64 150/157 2023-03-24T23:46:35.883 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : libfdisk-2.32.1-38.el8.x86_64 151/157 2023-03-24T23:46:36.100 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libfdisk-2.32.1-38.el8.x86_64 151/157 2023-03-24T23:46:36.101 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : libmount-2.32.1-38.el8.x86_64 152/157 2023-03-24T23:46:36.317 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libmount-2.32.1-38.el8.x86_64 152/157 2023-03-24T23:46:36.317 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : libblkid-2.32.1-38.el8.x86_64 153/157 2023-03-24T23:46:36.525 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libblkid-2.32.1-38.el8.x86_64 153/157 2023-03-24T23:46:36.526 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : libuuid-2.32.1-38.el8.x86_64 154/157 2023-03-24T23:46:36.742 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libuuid-2.32.1-38.el8.x86_64 154/157 2023-03-24T23:46:36.742 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : libsmartcols-2.32.1-38.el8.x86_64 155/157 2023-03-24T23:46:36.942 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libsmartcols-2.32.1-38.el8.x86_64 155/157 2023-03-24T23:46:37.117 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : elfutils-libelf-0.187-4.el8.x86_64 156/157 2023-03-24T23:46:37.118 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : openssl-libs-1:1.1.1k-7.el8.x86_64 157/157 2023-03-24T23:46:38.074 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: openssl-libs-1:1.1.1k-7.el8.x86_64 157/157 2023-03-24T23:46:38.074 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-2:18.0.0-3124.gc19e8f2a.el8.x86_64 1/157 2023-03-24T23:46:38.075 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-base-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2/157 2023-03-24T23:46:38.075 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-common-2:18.0.0-3124.gc19e8f2a.el8.x86_64 3/157 2023-03-24T23:46:38.075 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-fuse-2:18.0.0-3124.gc19e8f2a.el8.x86_64 4/157 2023-03-24T23:46:38.075 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-3124.gc19e8 5/157 2023-03-24T23:46:38.075 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mds-2:18.0.0-3124.gc19e8f2a.el8.x86_64 6/157 2023-03-24T23:46:38.075 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-2:18.0.0-3124.gc19e8f2a.el8.x86_64 7/157 2023-03-24T23:46:38.075 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mon-2:18.0.0-3124.gc19e8f2a.el8.x86_64 8/157 2023-03-24T23:46:38.076 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-osd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 9/157 2023-03-24T23:46:38.076 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-radosgw-2:18.0.0-3124.gc19e8f2a.el8.x86_64 10/157 2023-03-24T23:46:38.076 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-selinux-2:18.0.0-3124.gc19e8f2a.el8.x86_64 11/157 2023-03-24T23:46:38.076 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-test-2:18.0.0-3124.gc19e8f2a.el8.x86_64 12/157 2023-03-24T23:46:38.076 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : cephfs-mirror-2:18.0.0-3124.gc19e8f2a.el8.x86_64 13/157 2023-03-24T23:46:38.076 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libcephfs-devel-2:18.0.0-3124.gc19e8f2a.el8.x86_ 14/157 2023-03-24T23:46:38.076 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libcephfs2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 15/157 2023-03-24T23:46:38.076 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libcephsqlite-2:18.0.0-3124.gc19e8f2a.el8.x86_64 16/157 2023-03-24T23:46:38.077 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librados-devel-2:18.0.0-3124.gc19e8f2a.el8.x86_6 17/157 2023-03-24T23:46:38.077 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libradosstriper1-2:18.0.0-3124.gc19e8f2a.el8.x86 18/157 2023-03-24T23:46:38.078 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librgw2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 19/157 2023-03-24T23:46:38.078 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-ceph-argparse-2:18.0.0-3124.gc19e8f2a.el 20/157 2023-03-24T23:46:38.078 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-ceph-common-2:18.0.0-3124.gc19e8f2a.el8. 21/157 2023-03-24T23:46:38.078 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cephfs-2:18.0.0-3124.gc19e8f2a.el8.x86_6 22/157 2023-03-24T23:46:38.078 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rados-2:18.0.0-3124.gc19e8f2a.el8.x86_64 23/157 2023-03-24T23:46:38.079 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rbd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 24/157 2023-03-24T23:46:38.079 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rgw-2:18.0.0-3124.gc19e8f2a.el8.x86_64 25/157 2023-03-24T23:46:38.079 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : rbd-fuse-2:18.0.0-3124.gc19e8f2a.el8.x86_64 26/157 2023-03-24T23:46:38.079 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : rbd-mirror-2:18.0.0-3124.gc19e8f2a.el8.x86_64 27/157 2023-03-24T23:46:38.079 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : rbd-nbd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 28/157 2023-03-24T23:46:38.087 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-3124.gc19e8f2a. 29/157 2023-03-24T23:46:38.087 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-3124.gc19e8f2a.el8.noa 30/157 2023-03-24T23:46:38.087 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-3124.gc19e8f2a.el8.n 31/157 2023-03-24T23:46:38.087 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-3124.gc19 32/157 2023-03-24T23:46:38.088 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-3124.gc19e8f2a.el 33/157 2023-03-24T23:46:38.088 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-rook-2:18.0.0-3124.gc19e8f2a.el8.noarch 34/157 2023-03-24T23:46:38.088 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-3124.gc19e8f2a.e 35/157 2023-03-24T23:46:38.088 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-volume-2:18.0.0-3124.gc19e8f2a.el8.noarch 36/157 2023-03-24T23:46:38.088 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : cephadm-2:18.0.0-3124.gc19e8f2a.el8.noarch 37/157 2023-03-24T23:46:38.088 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : cephfs-top-2:18.0.0-3124.gc19e8f2a.el8.noarch 38/157 2023-03-24T23:46:38.088 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : automake-1.16.1-7.el8.noarch 39/157 2023-03-24T23:46:38.088 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : bison-3.0.4-10.el8.x86_64 40/157 2023-03-24T23:46:38.089 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : flex-2.6.1-9.el8.x86_64 41/157 2023-03-24T23:46:38.089 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : jq-1.6-6.el8.x86_64 42/157 2023-03-24T23:46:38.089 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libpq-13.5-1.el8.x86_64 43/157 2023-03-24T23:46:38.089 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 44/157 2023-03-24T23:46:38.089 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 45/157 2023-03-24T23:46:38.089 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 46/157 2023-03-24T23:46:38.089 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 47/157 2023-03-24T23:46:38.090 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 48/157 2023-03-24T23:46:38.090 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 49/157 2023-03-24T23:46:38.090 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : postgresql-10.23-1.module_el8+300+2c53004c.x86_6 50/157 2023-03-24T23:46:38.090 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : postgresql-contrib-10.23-1.module_el8+300+2c5300 51/157 2023-03-24T23:46:38.090 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : postgresql-server-10.23-1.module_el8+300+2c53004 52/157 2023-03-24T23:46:38.090 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 53/157 2023-03-24T23:46:38.090 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 54/157 2023-03-24T23:46:38.090 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 55/157 2023-03-24T23:46:38.091 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 56/157 2023-03-24T23:46:38.091 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 57/157 2023-03-24T23:46:38.091 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 58/157 2023-03-24T23:46:38.091 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 59/157 2023-03-24T23:46:38.091 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 60/157 2023-03-24T23:46:38.091 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 61/157 2023-03-24T23:46:38.091 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 62/157 2023-03-24T23:46:38.092 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 63/157 2023-03-24T23:46:38.092 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 64/157 2023-03-24T23:46:38.092 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 65/157 2023-03-24T23:46:38.092 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 66/157 2023-03-24T23:46:38.092 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 67/157 2023-03-24T23:46:38.092 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 68/157 2023-03-24T23:46:38.092 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 69/157 2023-03-24T23:46:38.092 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 70/157 2023-03-24T23:46:38.093 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : elfutils-libelf-devel-0.188-3.el8.x86_64 71/157 2023-03-24T23:46:38.093 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : fuse-2.9.7-16.el8.x86_64 72/157 2023-03-24T23:46:38.093 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 73/157 2023-03-24T23:46:38.093 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : krb5-devel-1.18.2-22.el8.x86_64 74/157 2023-03-24T23:46:38.093 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 75/157 2023-03-24T23:46:38.093 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libcom_err-devel-1.45.6-5.el8.x86_64 76/157 2023-03-24T23:46:38.093 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libgfortran-8.5.0-18.el8.x86_64 77/157 2023-03-24T23:46:38.093 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libquadmath-8.5.0-18.el8.x86_64 78/157 2023-03-24T23:46:38.094 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 79/157 2023-03-24T23:46:38.094 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libselinux-devel-2.9-6.el8.x86_64 80/157 2023-03-24T23:46:38.094 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 81/157 2023-03-24T23:46:38.094 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libuuid-devel-2.32.1-41.el8.x86_64 82/157 2023-03-24T23:46:38.094 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libverto-devel-0.3.2-2.el8.x86_64 83/157 2023-03-24T23:46:38.094 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openssl-devel-1:1.1.1k-9.el8.x86_64 84/157 2023-03-24T23:46:38.094 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : pcre2-devel-10.32-3.el8.x86_64 85/157 2023-03-24T23:46:38.094 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : pcre2-utf16-10.32-3.el8.x86_64 86/157 2023-03-24T23:46:38.095 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : pcre2-utf32-10.32-3.el8.x86_64 87/157 2023-03-24T23:46:38.095 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 88/157 2023-03-24T23:46:38.095 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 89/157 2023-03-24T23:46:38.095 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 90/157 2023-03-24T23:46:38.095 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 91/157 2023-03-24T23:46:38.095 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 92/157 2023-03-24T23:46:38.095 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 93/157 2023-03-24T23:46:38.095 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 94/157 2023-03-24T23:46:38.096 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : zlib-devel-1.2.11-19.el8.x86_64 95/157 2023-03-24T23:46:38.096 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 96/157 2023-03-24T23:46:38.096 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 97/157 2023-03-24T23:46:38.096 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 98/157 2023-03-24T23:46:38.096 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 99/157 2023-03-24T23:46:38.096 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 100/157 2023-03-24T23:46:38.096 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 101/157 2023-03-24T23:46:38.097 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 102/157 2023-03-24T23:46:38.097 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 103/157 2023-03-24T23:46:38.097 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 104/157 2023-03-24T23:46:38.097 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 105/157 2023-03-24T23:46:38.097 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 106/157 2023-03-24T23:46:38.097 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 107/157 2023-03-24T23:46:38.097 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 108/157 2023-03-24T23:46:38.097 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 109/157 2023-03-24T23:46:38.098 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 110/157 2023-03-24T23:46:38.098 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 111/157 2023-03-24T23:46:38.098 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 112/157 2023-03-24T23:46:38.098 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 113/157 2023-03-24T23:46:38.098 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-portend-2.6-1.el8.noarch 114/157 2023-03-24T23:46:38.098 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 115/157 2023-03-24T23:46:38.098 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 116/157 2023-03-24T23:46:38.098 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 117/157 2023-03-24T23:46:38.099 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 118/157 2023-03-24T23:46:38.099 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 119/157 2023-03-24T23:46:38.099 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 120/157 2023-03-24T23:46:38.099 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 121/157 2023-03-24T23:46:38.099 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 122/157 2023-03-24T23:46:38.099 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 123/157 2023-03-24T23:46:38.099 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 124/157 2023-03-24T23:46:38.099 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 125/157 2023-03-24T23:46:38.100 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 126/157 2023-03-24T23:46:38.100 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 127/157 2023-03-24T23:46:38.100 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librados2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 128/157 2023-03-24T23:46:38.100 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 129/157 2023-03-24T23:46:38.100 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librbd1-2:18.0.0-3124.gc19e8f2a.el8.x86_64 130/157 2023-03-24T23:46:38.100 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 131/157 2023-03-24T23:46:38.100 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-1.el 132/157 2023-03-24T23:46:38.101 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : NetworkManager-initscripts-updown-1:1.39.12-1.el 133/157 2023-03-24T23:46:38.101 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : elfutils-debuginfod-client-0.188-3.el8.x86_64 134/157 2023-03-24T23:46:38.101 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : elfutils-debuginfod-client-0.187-4.el8.x86_64 135/157 2023-03-24T23:46:38.101 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : elfutils-libelf-0.188-3.el8.x86_64 136/157 2023-03-24T23:46:38.101 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : elfutils-libelf-0.187-4.el8.x86_64 137/157 2023-03-24T23:46:38.101 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : elfutils-libs-0.188-3.el8.x86_64 138/157 2023-03-24T23:46:38.101 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : elfutils-libs-0.187-4.el8.x86_64 139/157 2023-03-24T23:46:38.101 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libblkid-2.32.1-41.el8.x86_64 140/157 2023-03-24T23:46:38.102 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libblkid-2.32.1-38.el8.x86_64 141/157 2023-03-24T23:46:38.102 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libfdisk-2.32.1-41.el8.x86_64 142/157 2023-03-24T23:46:38.102 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libfdisk-2.32.1-38.el8.x86_64 143/157 2023-03-24T23:46:38.102 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libmount-2.32.1-41.el8.x86_64 144/157 2023-03-24T23:46:38.102 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libmount-2.32.1-38.el8.x86_64 145/157 2023-03-24T23:46:38.102 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libsmartcols-2.32.1-41.el8.x86_64 146/157 2023-03-24T23:46:38.102 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libsmartcols-2.32.1-38.el8.x86_64 147/157 2023-03-24T23:46:38.102 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libuuid-2.32.1-41.el8.x86_64 148/157 2023-03-24T23:46:38.103 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libuuid-2.32.1-38.el8.x86_64 149/157 2023-03-24T23:46:38.103 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openssl-1:1.1.1k-9.el8.x86_64 150/157 2023-03-24T23:46:38.103 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openssl-1:1.1.1k-7.el8.x86_64 151/157 2023-03-24T23:46:38.103 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openssl-libs-1:1.1.1k-9.el8.x86_64 152/157 2023-03-24T23:46:38.103 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openssl-libs-1:1.1.1k-7.el8.x86_64 153/157 2023-03-24T23:46:38.103 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : util-linux-2.32.1-41.el8.x86_64 154/157 2023-03-24T23:46:38.103 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : util-linux-2.32.1-38.el8.x86_64 155/157 2023-03-24T23:46:38.103 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : util-linux-user-2.32.1-41.el8.x86_64 156/157 2023-03-24T23:46:40.171 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : util-linux-user-2.32.1-38.el8.x86_64 157/157 2023-03-24T23:46:40.171 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:46:40.172 INFO:teuthology.orchestra.run.smithi072.stdout:Upgraded: 2023-03-24T23:46:40.172 INFO:teuthology.orchestra.run.smithi072.stdout: NetworkManager-initscripts-updown-1:1.40.16-1.el8.noarch 2023-03-24T23:46:40.172 INFO:teuthology.orchestra.run.smithi072.stdout: elfutils-debuginfod-client-0.188-3.el8.x86_64 2023-03-24T23:46:40.172 INFO:teuthology.orchestra.run.smithi072.stdout: elfutils-libelf-0.188-3.el8.x86_64 2023-03-24T23:46:40.172 INFO:teuthology.orchestra.run.smithi072.stdout: elfutils-libs-0.188-3.el8.x86_64 2023-03-24T23:46:40.172 INFO:teuthology.orchestra.run.smithi072.stdout: libblkid-2.32.1-41.el8.x86_64 2023-03-24T23:46:40.173 INFO:teuthology.orchestra.run.smithi072.stdout: libfdisk-2.32.1-41.el8.x86_64 2023-03-24T23:46:40.173 INFO:teuthology.orchestra.run.smithi072.stdout: libmount-2.32.1-41.el8.x86_64 2023-03-24T23:46:40.173 INFO:teuthology.orchestra.run.smithi072.stdout: librados2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:40.173 INFO:teuthology.orchestra.run.smithi072.stdout: librbd1-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:40.173 INFO:teuthology.orchestra.run.smithi072.stdout: libsmartcols-2.32.1-41.el8.x86_64 2023-03-24T23:46:40.173 INFO:teuthology.orchestra.run.smithi072.stdout: libuuid-2.32.1-41.el8.x86_64 2023-03-24T23:46:40.173 INFO:teuthology.orchestra.run.smithi072.stdout: openssl-1:1.1.1k-9.el8.x86_64 2023-03-24T23:46:40.174 INFO:teuthology.orchestra.run.smithi072.stdout: openssl-libs-1:1.1.1k-9.el8.x86_64 2023-03-24T23:46:40.174 INFO:teuthology.orchestra.run.smithi072.stdout: util-linux-2.32.1-41.el8.x86_64 2023-03-24T23:46:40.174 INFO:teuthology.orchestra.run.smithi072.stdout: util-linux-user-2.32.1-41.el8.x86_64 2023-03-24T23:46:40.174 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-03-24T23:46:40.174 INFO:teuthology.orchestra.run.smithi072.stdout: automake-1.16.1-7.el8.noarch 2023-03-24T23:46:40.174 INFO:teuthology.orchestra.run.smithi072.stdout: bison-3.0.4-10.el8.x86_64 2023-03-24T23:46:40.174 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:40.175 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-base-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:40.175 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-common-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:40.175 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-fuse-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:40.175 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-grafana-dashboards-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T23:46:40.175 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-immutable-object-cache-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:40.175 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mds-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:40.175 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:40.176 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-cephadm-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T23:46:40.176 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-dashboard-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T23:46:40.176 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-diskprediction-local-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T23:46:40.176 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-modules-core-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T23:46:40.176 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-rook-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T23:46:40.176 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mon-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:40.176 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-osd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:40.176 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-prometheus-alerts-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T23:46:40.177 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-radosgw-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:40.177 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-selinux-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:40.177 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-test-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:40.177 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-volume-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T23:46:40.177 INFO:teuthology.orchestra.run.smithi072.stdout: cephadm-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T23:46:40.177 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-mirror-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:40.177 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-top-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T23:46:40.178 INFO:teuthology.orchestra.run.smithi072.stdout: elfutils-libelf-devel-0.188-3.el8.x86_64 2023-03-24T23:46:40.178 INFO:teuthology.orchestra.run.smithi072.stdout: flex-2.6.1-9.el8.x86_64 2023-03-24T23:46:40.178 INFO:teuthology.orchestra.run.smithi072.stdout: fuse-2.9.7-16.el8.x86_64 2023-03-24T23:46:40.178 INFO:teuthology.orchestra.run.smithi072.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-03-24T23:46:40.178 INFO:teuthology.orchestra.run.smithi072.stdout: jq-1.6-6.el8.x86_64 2023-03-24T23:46:40.178 INFO:teuthology.orchestra.run.smithi072.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-03-24T23:46:40.178 INFO:teuthology.orchestra.run.smithi072.stdout: krb5-devel-1.18.2-22.el8.x86_64 2023-03-24T23:46:40.178 INFO:teuthology.orchestra.run.smithi072.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-03-24T23:46:40.179 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs-devel-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:40.179 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:40.179 INFO:teuthology.orchestra.run.smithi072.stdout: libcephsqlite-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:40.179 INFO:teuthology.orchestra.run.smithi072.stdout: libcom_err-devel-1.45.6-5.el8.x86_64 2023-03-24T23:46:40.179 INFO:teuthology.orchestra.run.smithi072.stdout: libgfortran-8.5.0-18.el8.x86_64 2023-03-24T23:46:40.179 INFO:teuthology.orchestra.run.smithi072.stdout: liboath-2.6.2-3.el8.x86_64 2023-03-24T23:46:40.179 INFO:teuthology.orchestra.run.smithi072.stdout: libpq-13.5-1.el8.x86_64 2023-03-24T23:46:40.180 INFO:teuthology.orchestra.run.smithi072.stdout: libquadmath-8.5.0-18.el8.x86_64 2023-03-24T23:46:40.180 INFO:teuthology.orchestra.run.smithi072.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-03-24T23:46:40.180 INFO:teuthology.orchestra.run.smithi072.stdout: librados-devel-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:40.180 INFO:teuthology.orchestra.run.smithi072.stdout: libradosstriper1-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:40.180 INFO:teuthology.orchestra.run.smithi072.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-03-24T23:46:40.180 INFO:teuthology.orchestra.run.smithi072.stdout: librgw2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:40.181 INFO:teuthology.orchestra.run.smithi072.stdout: libselinux-devel-2.9-6.el8.x86_64 2023-03-24T23:46:40.181 INFO:teuthology.orchestra.run.smithi072.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-03-24T23:46:40.181 INFO:teuthology.orchestra.run.smithi072.stdout: libtool-2.4.6-25.el8.x86_64 2023-03-24T23:46:40.181 INFO:teuthology.orchestra.run.smithi072.stdout: libunwind-1.3.1-3.el8.x86_64 2023-03-24T23:46:40.181 INFO:teuthology.orchestra.run.smithi072.stdout: libuuid-devel-2.32.1-41.el8.x86_64 2023-03-24T23:46:40.181 INFO:teuthology.orchestra.run.smithi072.stdout: libverto-devel-0.3.2-2.el8.x86_64 2023-03-24T23:46:40.181 INFO:teuthology.orchestra.run.smithi072.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-03-24T23:46:40.182 INFO:teuthology.orchestra.run.smithi072.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-03-24T23:46:40.182 INFO:teuthology.orchestra.run.smithi072.stdout: openblas-0.3.15-4.el8.x86_64 2023-03-24T23:46:40.182 INFO:teuthology.orchestra.run.smithi072.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-03-24T23:46:40.182 INFO:teuthology.orchestra.run.smithi072.stdout: openssl-devel-1:1.1.1k-9.el8.x86_64 2023-03-24T23:46:40.182 INFO:teuthology.orchestra.run.smithi072.stdout: pcre2-devel-10.32-3.el8.x86_64 2023-03-24T23:46:40.182 INFO:teuthology.orchestra.run.smithi072.stdout: pcre2-utf16-10.32-3.el8.x86_64 2023-03-24T23:46:40.182 INFO:teuthology.orchestra.run.smithi072.stdout: pcre2-utf32-10.32-3.el8.x86_64 2023-03-24T23:46:40.183 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-10.23-1.module_el8+300+2c53004c.x86_64 2023-03-24T23:46:40.183 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-contrib-10.23-1.module_el8+300+2c53004c.x86_64 2023-03-24T23:46:40.183 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-server-10.23-1.module_el8+300+2c53004c.x86_64 2023-03-24T23:46:40.183 INFO:teuthology.orchestra.run.smithi072.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-03-24T23:46:40.183 INFO:teuthology.orchestra.run.smithi072.stdout: python3-babel-2.5.1-7.el8.noarch 2023-03-24T23:46:40.183 INFO:teuthology.orchestra.run.smithi072.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-03-24T23:46:40.183 INFO:teuthology.orchestra.run.smithi072.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-03-24T23:46:40.183 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-03-24T23:46:40.184 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-argparse-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:40.184 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-common-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:40.184 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cephfs-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:40.184 INFO:teuthology.orchestra.run.smithi072.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-03-24T23:46:40.184 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-03-24T23:46:40.184 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-03-24T23:46:40.184 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-03-24T23:46:40.185 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-03-24T23:46:40.185 INFO:teuthology.orchestra.run.smithi072.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-03-24T23:46:40.185 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-6.2-6.el8.noarch 2023-03-24T23:46:40.185 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-03-24T23:46:40.185 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-03-24T23:46:40.185 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-03-24T23:46:40.185 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-03-24T23:46:40.185 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-03-24T23:46:40.186 INFO:teuthology.orchestra.run.smithi072.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-03-24T23:46:40.186 INFO:teuthology.orchestra.run.smithi072.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-03-24T23:46:40.186 INFO:teuthology.orchestra.run.smithi072.stdout: python3-mako-1.0.6-14.el8.noarch 2023-03-24T23:46:40.186 INFO:teuthology.orchestra.run.smithi072.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-03-24T23:46:40.186 INFO:teuthology.orchestra.run.smithi072.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-03-24T23:46:40.186 INFO:teuthology.orchestra.run.smithi072.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-03-24T23:46:40.186 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-03-24T23:46:40.187 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-03-24T23:46:40.187 INFO:teuthology.orchestra.run.smithi072.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-03-24T23:46:40.187 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-03-24T23:46:40.187 INFO:teuthology.orchestra.run.smithi072.stdout: python3-portend-2.6-1.el8.noarch 2023-03-24T23:46:40.187 INFO:teuthology.orchestra.run.smithi072.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-03-24T23:46:40.187 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-03-24T23:46:40.187 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-03-24T23:46:40.187 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-03-24T23:46:40.188 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pycparser-2.14-14.el8.noarch 2023-03-24T23:46:40.188 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pytz-2017.2-9.el8.noarch 2023-03-24T23:46:40.188 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rados-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:40.188 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rbd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:40.188 INFO:teuthology.orchestra.run.smithi072.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-03-24T23:46:40.188 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-03-24T23:46:40.188 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rgw-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:40.188 INFO:teuthology.orchestra.run.smithi072.stdout: python3-routes-2.4.1-12.el8.noarch 2023-03-24T23:46:40.189 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rsa-4.8-1.el8.noarch 2023-03-24T23:46:40.189 INFO:teuthology.orchestra.run.smithi072.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-03-24T23:46:40.189 INFO:teuthology.orchestra.run.smithi072.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-03-24T23:46:40.189 INFO:teuthology.orchestra.run.smithi072.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-03-24T23:46:40.189 INFO:teuthology.orchestra.run.smithi072.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-03-24T23:46:40.189 INFO:teuthology.orchestra.run.smithi072.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-03-24T23:46:40.189 INFO:teuthology.orchestra.run.smithi072.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-03-24T23:46:40.189 INFO:teuthology.orchestra.run.smithi072.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-03-24T23:46:40.190 INFO:teuthology.orchestra.run.smithi072.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-03-24T23:46:40.190 INFO:teuthology.orchestra.run.smithi072.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-03-24T23:46:40.190 INFO:teuthology.orchestra.run.smithi072.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-03-24T23:46:40.190 INFO:teuthology.orchestra.run.smithi072.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-03-24T23:46:40.190 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-fuse-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:40.190 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-mirror-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:40.190 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-nbd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T23:46:40.191 INFO:teuthology.orchestra.run.smithi072.stdout: socat-1.7.4.1-1.el8.x86_64 2023-03-24T23:46:40.191 INFO:teuthology.orchestra.run.smithi072.stdout: thrift-0.13.0-2.el8.x86_64 2023-03-24T23:46:40.191 INFO:teuthology.orchestra.run.smithi072.stdout: uuid-1.6.2-43.el8.x86_64 2023-03-24T23:46:40.191 INFO:teuthology.orchestra.run.smithi072.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-03-24T23:46:40.191 INFO:teuthology.orchestra.run.smithi072.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-03-24T23:46:40.192 INFO:teuthology.orchestra.run.smithi072.stdout: zlib-devel-1.2.11-19.el8.x86_64 2023-03-24T23:46:40.192 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:46:40.192 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-03-24T23:46:40.543 DEBUG:teuthology.parallel:result is None 2023-03-24T23:46:40.545 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=c19e8f2a497e95f75abc43ec14bc6095e30dd2cb 2023-03-24T23:46:40.682 DEBUG:teuthology.orchestra.run.smithi072:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-03-24T23:46:40.706 INFO:teuthology.orchestra.run.smithi072.stdout:18.0.0-3124.gc19e8f2a.el8 2023-03-24T23:46:40.707 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-3124.gc19e8f2a.el8 2023-03-24T23:46:40.707 INFO:teuthology.task.install:The correct ceph version 18.0.0-3124.gc19e8f2a is installed. 2023-03-24T23:46:40.708 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=c19e8f2a497e95f75abc43ec14bc6095e30dd2cb 2023-03-24T23:46:40.841 DEBUG:teuthology.orchestra.run.smithi146:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-03-24T23:46:40.867 INFO:teuthology.orchestra.run.smithi146.stdout:18.0.0-3124.gc19e8f2a.el8 2023-03-24T23:46:40.867 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-3124.gc19e8f2a.el8 2023-03-24T23:46:40.868 INFO:teuthology.task.install:The correct ceph version 18.0.0-3124.gc19e8f2a is installed. 2023-03-24T23:46:40.868 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-03-24T23:46:40.869 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-24T23:46:40.869 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-03-24T23:46:40.900 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-03-24T23:46:40.901 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-03-24T23:46:40.939 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-03-24T23:46:40.939 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-24T23:46:40.940 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/usr/bin/daemon-helper 2023-03-24T23:46:40.968 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-03-24T23:46:41.035 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-03-24T23:46:41.036 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/daemon-helper 2023-03-24T23:46:41.064 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-03-24T23:46:41.131 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-03-24T23:46:41.132 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-24T23:46:41.132 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/usr/bin/adjust-ulimits 2023-03-24T23:46:41.159 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-03-24T23:46:41.227 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-03-24T23:46:41.227 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/adjust-ulimits 2023-03-24T23:46:41.254 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-03-24T23:46:41.322 INFO:teuthology.run_tasks:Running task ceph... 2023-03-24T23:46:41.399 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-03-24T23:46:41.400 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 777 /var/log/ceph 2023-03-24T23:46:41.402 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 777 /var/log/ceph 2023-03-24T23:46:41.429 INFO:tasks.ceph:Disabling ceph logrotate... 2023-03-24T23:46:41.429 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -f -- /etc/logrotate.d/ceph 2023-03-24T23:46:41.470 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f -- /etc/logrotate.d/ceph 2023-03-24T23:46:41.496 INFO:tasks.ceph:Creating extra log directories... 2023-03-24T23:46:41.497 DEBUG:teuthology.orchestra.run.smithi072:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-03-24T23:46:41.537 DEBUG:teuthology.orchestra.run.smithi146:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-03-24T23:46:41.570 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-03-24T23:46:41.571 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-03-24T23:46:41.571 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-03-24T23:46:41.571 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-24T23:46:41.572 DEBUG:teuthology.orchestra.run.smithi072:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-03-24T23:46:41.621 DEBUG:teuthology.orchestra.run.smithi072:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-03-24T23:46:41.738 DEBUG:teuthology.orchestra.run.smithi072:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-03-24T23:46:41.765 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-03-24T23:46:41.766 DEBUG:teuthology.orchestra.run.smithi146:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-03-24T23:46:41.784 DEBUG:teuthology.orchestra.run.smithi146:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-03-24T23:46:41.902 DEBUG:teuthology.orchestra.run.smithi146:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-03-24T23:46:41.935 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-03-24T23:46:41.935 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'unmatched fragstat', 'unmatched rstat', 'was unreadable, recreating it now', 'Scrub error on inode', 'Metadata damage detected', 'inconsistent rstat on inode', 'Error recovering journal'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-03-24T23:46:41.935 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2023-03-24_23:11:30-fs-wip-pdonnell-testing-20230323.162417-distro-default-smithi/7219632', 'branch': 'wip-pdonnell-testing-20230323.162417', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/data-scan}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7219632', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'pdonnell-2023-03-24_23:11:30-fs-wip-pdonnell-testing-20230323.162417-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '8.stream', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20230323.162417'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'unmatched fragstat', 'unmatched rstat', 'was unreadable, recreating it now', 'Scrub error on inode', 'Metadata damage detected', 'inconsistent rstat on inode', 'Error recovering journal'], 'sha1': 'c19e8f2a497e95f75abc43ec14bc6095e30dd2cb'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'c19e8f2a497e95f75abc43ec14bc6095e30dd2cb'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20230323.162417', 'sha1': '1e7de2f05230c7419acf5ca7f5fd22964e30ab77'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 55, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'sha1': 'c19e8f2a497e95f75abc43ec14bc6095e30dd2cb', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20230323.162417', 'suite_path': '/home/teuthworker/src/github.com_batrick_ceph_1e7de2f05230c7419acf5ca7f5fd22964e30ab77/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/batrick/ceph.git', 'suite_sha1': '1e7de2f05230c7419acf5ca7f5fd22964e30ab77', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi072.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwaltmMfrsNs7H/qyiEC7tdKlCS2yuw0lbX5+T5Gw7rzxvOkFR7S+V/P43cwZ3fPw+pzl6kG3TstpXwZ3YRQBi0mDatyWptWVfOgqQfvj6C1kGezTtZ678cV4ZKOmzq+dH1Idi87dKTpmT8AF0Ghm2poDrj3ommn6jN2ldA/30808GTS2s6O67cne9qP513dQzBKvVmxH1Da37nkwgddYnP7BlaWgNzhZNM95l2zkVnhxlRYjyNrWxdpXCCOPs+BafbAQQNaM0SG5CVCHgn01ehouk1J5CZVbO8QosFGW9plxdz+npUxxU6YZD/PrSEEhjURwv2y8SYlP+kN/t09eJvfltdPA98g+IMmq0MpEyURirDRiZyjKcBOwb//I9Rm2B5ntWJPj0bRatPyIiteMNwMHO0kA1+UIjVAotUrvKFn/S67brBe3RCE42OagZNjAan6/9ZUWcHcKFKtFWHew0ie6+xMHvyzejw44WZihJeWIwH9FI/psL6+E9YaYPIWc=', 'smithi146.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMkAiVyNxV/ORARn9WKQ1l5a9Vj0+9RCo5UfCigB6duoMmd7Bn90XfZOYN9e7x2VGlH+qHDjXfEJHtLLeNSKWdQ='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'c19e8f2a497e95f75abc43ec14bc6095e30dd2cb'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_data_scan']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'parallel-gzip', 'teuthology_sha1': 'fe9b6e52013911547168fef4a8506325b6c76139', 'timestamp': '2023-03-24_23:11:30', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.4538'} 2023-03-24T23:46:41.936 DEBUG:teuthology.orchestra.run.smithi072:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-03-24T23:46:41.939 DEBUG:teuthology.orchestra.run.smithi146:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-03-24T23:46:41.987 DEBUG:teuthology.orchestra.run.smithi072:> sudo install -d -m0777 -- /var/run/ceph 2023-03-24T23:46:41.995 DEBUG:teuthology.orchestra.run.smithi146:> sudo install -d -m0777 -- /var/run/ceph 2023-03-24T23:46:42.055 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-24T23:46:42.055 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/scratch_devs of=/dev/stdout 2023-03-24T23:46:42.079 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-03-24T23:46:42.080 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_1 2023-03-24T23:46:42.141 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-03-24T23:46:42.141 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-24T23:46:42.142 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 6h/6d Inode: 388218 Links: 1 2023-03-24T23:46:42.142 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-24T23:46:42.143 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2023-03-24T23:46:42.143 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2023-03-24 23:46:37.667992521 +0000 2023-03-24T23:46:42.143 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2023-03-24 23:43:48.738691067 +0000 2023-03-24T23:46:42.143 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2023-03-24 23:43:48.738691067 +0000 2023-03-24T23:46:42.144 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: - 2023-03-24T23:46:42.145 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-03-24T23:46:42.206 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2023-03-24T23:46:42.206 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2023-03-24T23:46:42.207 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000221296 s, 2.3 MB/s 2023-03-24T23:46:42.207 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-03-24T23:46:42.266 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_2 2023-03-24T23:46:42.326 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-03-24T23:46:42.327 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-24T23:46:42.327 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 6h/6d Inode: 384533 Links: 1 2023-03-24T23:46:42.327 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-24T23:46:42.327 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2023-03-24T23:46:42.327 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2023-03-24 23:46:37.667992521 +0000 2023-03-24T23:46:42.327 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2023-03-24 23:43:49.274679538 +0000 2023-03-24T23:46:42.327 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2023-03-24 23:43:49.274679538 +0000 2023-03-24T23:46:42.328 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: - 2023-03-24T23:46:42.328 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-03-24T23:46:42.393 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2023-03-24T23:46:42.394 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2023-03-24T23:46:42.394 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000213625 s, 2.4 MB/s 2023-03-24T23:46:42.395 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-03-24T23:46:42.453 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_3 2023-03-24T23:46:42.511 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-03-24T23:46:42.512 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-24T23:46:42.512 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 6h/6d Inode: 388071 Links: 1 2023-03-24T23:46:42.512 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-24T23:46:42.512 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2023-03-24T23:46:42.512 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2023-03-24 23:46:37.667992521 +0000 2023-03-24T23:46:42.512 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2023-03-24 23:43:49.795668332 +0000 2023-03-24T23:46:42.512 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2023-03-24 23:43:49.795668332 +0000 2023-03-24T23:46:42.513 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: - 2023-03-24T23:46:42.513 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-03-24T23:46:42.577 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2023-03-24T23:46:42.577 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2023-03-24T23:46:42.578 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000208547 s, 2.5 MB/s 2023-03-24T23:46:42.578 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-03-24T23:46:42.637 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_4 2023-03-24T23:46:42.696 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-03-24T23:46:42.696 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-24T23:46:42.696 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 6h/6d Inode: 388618 Links: 1 2023-03-24T23:46:42.697 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-24T23:46:42.697 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2023-03-24T23:46:42.697 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2023-03-24 23:46:37.667992521 +0000 2023-03-24T23:46:42.698 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2023-03-24 23:43:50.328656867 +0000 2023-03-24T23:46:42.698 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2023-03-24 23:43:50.328656867 +0000 2023-03-24T23:46:42.698 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: - 2023-03-24T23:46:42.698 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-03-24T23:46:42.761 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2023-03-24T23:46:42.762 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2023-03-24T23:46:42.762 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000185243 s, 2.8 MB/s 2023-03-24T23:46:42.763 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-03-24T23:46:42.821 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-03-24T23:46:42.822 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-03-24T23:46:42.822 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/scratch_devs of=/dev/stdout 2023-03-24T23:46:42.840 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-03-24T23:46:42.840 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_1 2023-03-24T23:46:42.898 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-03-24T23:46:42.898 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-24T23:46:42.898 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 6h/6d Inode: 388393 Links: 1 2023-03-24T23:46:42.899 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-24T23:46:42.899 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2023-03-24T23:46:42.899 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2023-03-24 23:46:30.443783990 +0000 2023-03-24T23:46:42.899 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2023-03-24 23:43:43.762540578 +0000 2023-03-24T23:46:42.899 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2023-03-24 23:43:43.762540578 +0000 2023-03-24T23:46:42.899 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: - 2023-03-24T23:46:42.899 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-03-24T23:46:42.964 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2023-03-24T23:46:42.964 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2023-03-24T23:46:42.965 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000237241 s, 2.2 MB/s 2023-03-24T23:46:42.966 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-03-24T23:46:43.025 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_2 2023-03-24T23:46:43.082 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-03-24T23:46:43.082 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-24T23:46:43.082 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 6h/6d Inode: 384722 Links: 1 2023-03-24T23:46:43.082 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-24T23:46:43.083 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2023-03-24T23:46:43.083 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2023-03-24 23:46:30.444783973 +0000 2023-03-24T23:46:43.083 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2023-03-24 23:43:44.283531931 +0000 2023-03-24T23:46:43.083 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2023-03-24 23:43:44.283531931 +0000 2023-03-24T23:46:43.083 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: - 2023-03-24T23:46:43.083 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-03-24T23:46:43.147 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2023-03-24T23:46:43.148 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2023-03-24T23:46:43.148 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000208572 s, 2.5 MB/s 2023-03-24T23:46:43.149 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-03-24T23:46:43.207 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_3 2023-03-24T23:46:43.265 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-03-24T23:46:43.266 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-24T23:46:43.266 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 6h/6d Inode: 390217 Links: 1 2023-03-24T23:46:43.266 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-24T23:46:43.266 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2023-03-24T23:46:43.266 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2023-03-24 23:46:30.444783973 +0000 2023-03-24T23:46:43.266 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2023-03-24 23:43:44.801523334 +0000 2023-03-24T23:46:43.266 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2023-03-24 23:43:44.801523334 +0000 2023-03-24T23:46:43.267 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: - 2023-03-24T23:46:43.267 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-03-24T23:46:43.331 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2023-03-24T23:46:43.332 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2023-03-24T23:46:43.332 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000217463 s, 2.4 MB/s 2023-03-24T23:46:43.333 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-03-24T23:46:43.392 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_4 2023-03-24T23:46:43.450 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-03-24T23:46:43.450 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-24T23:46:43.450 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 6h/6d Inode: 390325 Links: 1 2023-03-24T23:46:43.450 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-24T23:46:43.450 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2023-03-24T23:46:43.451 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2023-03-24 23:46:30.444783973 +0000 2023-03-24T23:46:43.451 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2023-03-24 23:43:45.328514587 +0000 2023-03-24T23:46:43.451 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2023-03-24 23:43:45.328514587 +0000 2023-03-24T23:46:43.451 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: - 2023-03-24T23:46:43.451 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-03-24T23:46:43.515 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2023-03-24T23:46:43.516 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2023-03-24T23:46:43.516 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000205427 s, 2.5 MB/s 2023-03-24T23:46:43.517 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-03-24T23:46:43.575 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-03-24T23:46:43.576 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi072.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi146.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-03-24T23:46:43.576 INFO:tasks.ceph:Generating config... 2023-03-24T23:46:43.577 INFO:tasks.ceph:[client] client mount timeout = 600 2023-03-24T23:46:43.577 INFO:tasks.ceph:[client] debug client = 20 2023-03-24T23:46:43.578 INFO:tasks.ceph:[client] debug ms = 1 2023-03-24T23:46:43.578 INFO:tasks.ceph:[client] pid file = 2023-03-24T23:46:43.578 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-03-24T23:46:43.578 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-03-24T23:46:43.578 INFO:tasks.ceph:[mds] debug mds = 20 2023-03-24T23:46:43.579 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-03-24T23:46:43.579 INFO:tasks.ceph:[mds] debug ms = 1 2023-03-24T23:46:43.579 INFO:tasks.ceph:[mds] mds debug frag = True 2023-03-24T23:46:43.579 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-03-24T23:46:43.579 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-03-24T23:46:43.580 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-03-24T23:46:43.580 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-03-24T23:46:43.580 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-03-24T23:46:43.580 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-03-24T23:46:43.580 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-03-24T23:46:43.581 INFO:tasks.ceph:[mgr] debug ms = 1 2023-03-24T23:46:43.581 INFO:tasks.ceph:[mon] debug mon = 20 2023-03-24T23:46:43.581 INFO:tasks.ceph:[mon] debug ms = 1 2023-03-24T23:46:43.581 INFO:tasks.ceph:[mon] debug paxos = 20 2023-03-24T23:46:43.581 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-03-24T23:46:43.582 INFO:tasks.ceph:[osd] bdev async discard = True 2023-03-24T23:46:43.582 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-03-24T23:46:43.582 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-03-24T23:46:43.582 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-03-24T23:46:43.582 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-03-24T23:46:43.582 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-03-24T23:46:43.583 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-03-24T23:46:43.583 INFO:tasks.ceph:[osd] debug ms = 1 2023-03-24T23:46:43.583 INFO:tasks.ceph:[osd] debug osd = 20 2023-03-24T23:46:43.583 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-03-24T23:46:43.583 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-03-24T23:46:43.584 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-03-24T23:46:43.584 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-03-24T23:46:43.584 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-03-24T23:46:43.584 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-03-24T23:46:43.584 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-03-24T23:46:43.585 INFO:tasks.ceph:Setting up mon.a... 2023-03-24T23:46:43.585 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-03-24T23:46:43.643 INFO:teuthology.orchestra.run.smithi072.stdout:creating /etc/ceph/ceph.keyring 2023-03-24T23:46:43.645 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-03-24T23:46:43.691 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-03-24T23:46:43.717 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.72'), ('mon.b', '172.21.15.146'), ('mon.c', '[v2:172.21.15.146:3301,v1:172.21.15.146:6790]')] 2023-03-24T23:46:43.718 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.72,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'false', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-03-24T23:46:43.719 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-24T23:46:43.719 DEBUG:teuthology.orchestra.run.smithi072:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-03-24T23:46:43.776 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.72 --add b 172.21.15.146 --addv c '[v2:172.21.15.146:3301,v1:172.21.15.146:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-03-24T23:46:43.866 INFO:teuthology.orchestra.run.smithi072.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-03-24T23:46:43.866 INFO:teuthology.orchestra.run.smithi072.stdout:monmaptool: generated fsid d3c122b7-f0b4-4c96-8f85-86f129e51880 2023-03-24T23:46:43.867 INFO:teuthology.orchestra.run.smithi072.stdout:setting min_mon_release = pacific 2023-03-24T23:46:43.867 INFO:teuthology.orchestra.run.smithi072.stdout:epoch 0 2023-03-24T23:46:43.867 INFO:teuthology.orchestra.run.smithi072.stdout:fsid d3c122b7-f0b4-4c96-8f85-86f129e51880 2023-03-24T23:46:43.867 INFO:teuthology.orchestra.run.smithi072.stdout:last_changed 2023-03-24T23:46:43.866631+0000 2023-03-24T23:46:43.867 INFO:teuthology.orchestra.run.smithi072.stdout:created 2023-03-24T23:46:43.866631+0000 2023-03-24T23:46:43.867 INFO:teuthology.orchestra.run.smithi072.stdout:min_mon_release 16 (pacific) 2023-03-24T23:46:43.867 INFO:teuthology.orchestra.run.smithi072.stdout:election_strategy: 1 2023-03-24T23:46:43.868 INFO:teuthology.orchestra.run.smithi072.stdout:0: [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] mon.a 2023-03-24T23:46:43.868 INFO:teuthology.orchestra.run.smithi072.stdout:1: [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] mon.b 2023-03-24T23:46:43.868 INFO:teuthology.orchestra.run.smithi072.stdout:2: [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] mon.c 2023-03-24T23:46:43.868 INFO:teuthology.orchestra.run.smithi072.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-03-24T23:46:43.869 DEBUG:teuthology.orchestra.run.smithi072:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-03-24T23:46:43.887 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID d3c122b7-f0b4-4c96-8f85-86f129e51880... 2023-03-24T23:46:43.888 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-03-24T23:46:43.929 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-03-24T23:46:43.972 INFO:teuthology.orchestra.run.smithi072.stdout:[global] 2023-03-24T23:46:43.972 INFO:teuthology.orchestra.run.smithi072.stdout: chdir = "" 2023-03-24T23:46:43.972 INFO:teuthology.orchestra.run.smithi072.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-03-24T23:46:43.972 INFO:teuthology.orchestra.run.smithi072.stdout: auth supported = cephx 2023-03-24T23:46:43.972 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:46:43.973 INFO:teuthology.orchestra.run.smithi072.stdout: filestore xattr use omap = true 2023-03-24T23:46:43.973 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:46:43.973 INFO:teuthology.orchestra.run.smithi072.stdout: mon clock drift allowed = 1.000 2023-03-24T23:46:43.973 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:46:43.973 INFO:teuthology.orchestra.run.smithi072.stdout: osd crush chooseleaf type = 0 2023-03-24T23:46:43.973 INFO:teuthology.orchestra.run.smithi072.stdout: auth debug = true 2023-03-24T23:46:43.973 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:46:43.974 INFO:teuthology.orchestra.run.smithi072.stdout: ms die on old message = true 2023-03-24T23:46:43.974 INFO:teuthology.orchestra.run.smithi072.stdout: ms die on bug = true 2023-03-24T23:46:43.974 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:46:43.974 INFO:teuthology.orchestra.run.smithi072.stdout: mon max pg per osd = 10000 # >= luminous 2023-03-24T23:46:43.974 INFO:teuthology.orchestra.run.smithi072.stdout: mon pg warn max object skew = 0 2023-03-24T23:46:43.974 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:46:43.974 INFO:teuthology.orchestra.run.smithi072.stdout: # disable pg_autoscaler by default for new pools 2023-03-24T23:46:43.974 INFO:teuthology.orchestra.run.smithi072.stdout: osd_pool_default_pg_autoscale_mode = off 2023-03-24T23:46:43.975 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:46:43.975 INFO:teuthology.orchestra.run.smithi072.stdout: osd pool default size = 2 2023-03-24T23:46:43.975 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:46:43.975 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd allow primary affinity = true 2023-03-24T23:46:43.975 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd allow pg remap = true 2023-03-24T23:46:43.975 INFO:teuthology.orchestra.run.smithi072.stdout: mon warn on legacy crush tunables = false 2023-03-24T23:46:43.975 INFO:teuthology.orchestra.run.smithi072.stdout: mon warn on crush straw calc version zero = false 2023-03-24T23:46:43.976 INFO:teuthology.orchestra.run.smithi072.stdout: mon warn on no sortbitwise = false 2023-03-24T23:46:43.976 INFO:teuthology.orchestra.run.smithi072.stdout: mon warn on osd down out interval zero = false 2023-03-24T23:46:43.976 INFO:teuthology.orchestra.run.smithi072.stdout: mon warn on too few osds = false 2023-03-24T23:46:43.976 INFO:teuthology.orchestra.run.smithi072.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-03-24T23:46:43.976 INFO:teuthology.orchestra.run.smithi072.stdout: mon_warn_on_pool_no_redundancy = false 2023-03-24T23:46:43.976 INFO:teuthology.orchestra.run.smithi072.stdout: mon_allow_pool_size_one = true 2023-03-24T23:46:43.976 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:46:43.976 INFO:teuthology.orchestra.run.smithi072.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-03-24T23:46:43.977 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:46:43.977 INFO:teuthology.orchestra.run.smithi072.stdout: osd default data pool replay window = 5 2023-03-24T23:46:43.977 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:46:43.977 INFO:teuthology.orchestra.run.smithi072.stdout: mon allow pool delete = true 2023-03-24T23:46:43.977 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:46:43.977 INFO:teuthology.orchestra.run.smithi072.stdout: mon cluster log file level = debug 2023-03-24T23:46:43.977 INFO:teuthology.orchestra.run.smithi072.stdout: debug asserts on shutdown = true 2023-03-24T23:46:43.978 INFO:teuthology.orchestra.run.smithi072.stdout: mon health detail to clog = false 2023-03-24T23:46:43.978 INFO:teuthology.orchestra.run.smithi072.stdout: mon host = "172.21.15.72,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]" 2023-03-24T23:46:43.978 INFO:teuthology.orchestra.run.smithi072.stdout: fsid = d3c122b7-f0b4-4c96-8f85-86f129e51880 2023-03-24T23:46:43.978 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:46:43.978 INFO:teuthology.orchestra.run.smithi072.stdout:[osd] 2023-03-24T23:46:43.978 INFO:teuthology.orchestra.run.smithi072.stdout: osd journal size = 100 2023-03-24T23:46:43.978 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:46:43.979 INFO:teuthology.orchestra.run.smithi072.stdout: osd scrub load threshold = 5.0 2023-03-24T23:46:43.979 INFO:teuthology.orchestra.run.smithi072.stdout: osd scrub max interval = 600 2023-03-24T23:46:43.979 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:46:43.979 INFO:teuthology.orchestra.run.smithi072.stdout: osd recover clone overlap = true 2023-03-24T23:46:43.979 INFO:teuthology.orchestra.run.smithi072.stdout: osd recovery max chunk = 1048576 2023-03-24T23:46:43.979 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:46:43.979 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug shutdown = true 2023-03-24T23:46:43.980 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug op order = false 2023-03-24T23:46:43.980 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug verify stray on activate = true 2023-03-24T23:46:43.980 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:46:43.980 INFO:teuthology.orchestra.run.smithi072.stdout: osd open classes on start = true 2023-03-24T23:46:43.980 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug pg log writeout = true 2023-03-24T23:46:43.980 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:46:43.980 INFO:teuthology.orchestra.run.smithi072.stdout: osd deep scrub update digest min age = 30 2023-03-24T23:46:43.981 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:46:43.981 INFO:teuthology.orchestra.run.smithi072.stdout: osd map max advance = 10 2023-03-24T23:46:43.981 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:46:43.981 INFO:teuthology.orchestra.run.smithi072.stdout: journal zero on create = true 2023-03-24T23:46:43.981 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:46:43.981 INFO:teuthology.orchestra.run.smithi072.stdout: filestore ondisk finisher threads = 3 2023-03-24T23:46:43.981 INFO:teuthology.orchestra.run.smithi072.stdout: filestore apply finisher threads = 3 2023-03-24T23:46:43.982 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:46:43.982 INFO:teuthology.orchestra.run.smithi072.stdout: bdev debug aio = true 2023-03-24T23:46:43.982 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug misdirected ops = true 2023-03-24T23:46:43.982 INFO:teuthology.orchestra.run.smithi072.stdout: bdev async discard = True 2023-03-24T23:46:43.982 INFO:teuthology.orchestra.run.smithi072.stdout: bdev enable discard = True 2023-03-24T23:46:43.982 INFO:teuthology.orchestra.run.smithi072.stdout: bluestore allocator = bitmap 2023-03-24T23:46:43.982 INFO:teuthology.orchestra.run.smithi072.stdout: bluestore block size = 96636764160 2023-03-24T23:46:43.982 INFO:teuthology.orchestra.run.smithi072.stdout: bluestore fsck on mount = True 2023-03-24T23:46:43.983 INFO:teuthology.orchestra.run.smithi072.stdout: debug bluefs = 1/20 2023-03-24T23:46:43.983 INFO:teuthology.orchestra.run.smithi072.stdout: debug bluestore = 1/20 2023-03-24T23:46:43.983 INFO:teuthology.orchestra.run.smithi072.stdout: debug ms = 1 2023-03-24T23:46:43.983 INFO:teuthology.orchestra.run.smithi072.stdout: debug osd = 20 2023-03-24T23:46:43.983 INFO:teuthology.orchestra.run.smithi072.stdout: debug rocksdb = 4/10 2023-03-24T23:46:43.983 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd backfillfull_ratio = 0.85 2023-03-24T23:46:43.983 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd full ratio = 0.9 2023-03-24T23:46:43.983 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd nearfull ratio = 0.8 2023-03-24T23:46:43.984 INFO:teuthology.orchestra.run.smithi072.stdout: osd failsafe full ratio = 0.95 2023-03-24T23:46:43.984 INFO:teuthology.orchestra.run.smithi072.stdout: osd objectstore = bluestore 2023-03-24T23:46:43.984 INFO:teuthology.orchestra.run.smithi072.stdout: osd op complaint time = 180 2023-03-24T23:46:43.984 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:46:43.984 INFO:teuthology.orchestra.run.smithi072.stdout:[mgr] 2023-03-24T23:46:43.984 INFO:teuthology.orchestra.run.smithi072.stdout: debug ms = 1 2023-03-24T23:46:43.984 INFO:teuthology.orchestra.run.smithi072.stdout: debug mgr = 20 2023-03-24T23:46:43.984 INFO:teuthology.orchestra.run.smithi072.stdout: debug mon = 20 2023-03-24T23:46:43.985 INFO:teuthology.orchestra.run.smithi072.stdout: debug auth = 20 2023-03-24T23:46:43.985 INFO:teuthology.orchestra.run.smithi072.stdout: mon reweight min pgs per osd = 4 2023-03-24T23:46:43.985 INFO:teuthology.orchestra.run.smithi072.stdout: mon reweight min bytes per osd = 10 2023-03-24T23:46:43.985 INFO:teuthology.orchestra.run.smithi072.stdout: mgr/telemetry/nag = false 2023-03-24T23:46:43.986 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:46:43.986 INFO:teuthology.orchestra.run.smithi072.stdout:[mon] 2023-03-24T23:46:43.986 INFO:teuthology.orchestra.run.smithi072.stdout: debug ms = 1 2023-03-24T23:46:43.986 INFO:teuthology.orchestra.run.smithi072.stdout: debug mon = 20 2023-03-24T23:46:43.986 INFO:teuthology.orchestra.run.smithi072.stdout: debug paxos = 20 2023-03-24T23:46:43.986 INFO:teuthology.orchestra.run.smithi072.stdout: debug auth = 20 2023-03-24T23:46:43.986 INFO:teuthology.orchestra.run.smithi072.stdout: mon data avail warn = 5 2023-03-24T23:46:43.986 INFO:teuthology.orchestra.run.smithi072.stdout: mon mgr mkfs grace = 240 2023-03-24T23:46:43.987 INFO:teuthology.orchestra.run.smithi072.stdout: mon reweight min pgs per osd = 4 2023-03-24T23:46:43.987 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd reporter subtree level = osd 2023-03-24T23:46:43.987 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd prime pg temp = true 2023-03-24T23:46:43.987 INFO:teuthology.orchestra.run.smithi072.stdout: mon reweight min bytes per osd = 10 2023-03-24T23:46:43.987 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:46:43.987 INFO:teuthology.orchestra.run.smithi072.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-03-24T23:46:43.987 INFO:teuthology.orchestra.run.smithi072.stdout: auth mon ticket ttl = 660 # 11m 2023-03-24T23:46:43.987 INFO:teuthology.orchestra.run.smithi072.stdout: auth service ticket ttl = 240 # 4m 2023-03-24T23:46:43.988 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:46:43.988 INFO:teuthology.orchestra.run.smithi072.stdout: # don't complain about insecure global_id in the test suite 2023-03-24T23:46:43.988 INFO:teuthology.orchestra.run.smithi072.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-03-24T23:46:43.988 INFO:teuthology.orchestra.run.smithi072.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-03-24T23:46:43.989 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:46:43.989 INFO:teuthology.orchestra.run.smithi072.stdout: # 1m isn't quite enough 2023-03-24T23:46:43.989 INFO:teuthology.orchestra.run.smithi072.stdout: mon_down_mkfs_grace = 2m 2023-03-24T23:46:43.989 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:46:43.990 INFO:teuthology.orchestra.run.smithi072.stdout: mon_warn_on_filestore_osds = false 2023-03-24T23:46:43.990 INFO:teuthology.orchestra.run.smithi072.stdout: mon op complaint time = 120 2023-03-24T23:46:43.990 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:46:43.991 INFO:teuthology.orchestra.run.smithi072.stdout:[client] 2023-03-24T23:46:43.991 INFO:teuthology.orchestra.run.smithi072.stdout: rgw cache enabled = true 2023-03-24T23:46:43.991 INFO:teuthology.orchestra.run.smithi072.stdout: rgw enable ops log = true 2023-03-24T23:46:43.991 INFO:teuthology.orchestra.run.smithi072.stdout: rgw enable usage log = true 2023-03-24T23:46:43.992 INFO:teuthology.orchestra.run.smithi072.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-03-24T23:46:43.992 INFO:teuthology.orchestra.run.smithi072.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-03-24T23:46:43.992 INFO:teuthology.orchestra.run.smithi072.stdout: client mount timeout = 600 2023-03-24T23:46:43.992 INFO:teuthology.orchestra.run.smithi072.stdout: debug client = 20 2023-03-24T23:46:43.993 INFO:teuthology.orchestra.run.smithi072.stdout: debug ms = 1 2023-03-24T23:46:43.993 INFO:teuthology.orchestra.run.smithi072.stdout: pid file = "" 2023-03-24T23:46:43.993 INFO:teuthology.orchestra.run.smithi072.stdout: rados mon op timeout = 900 2023-03-24T23:46:43.993 INFO:teuthology.orchestra.run.smithi072.stdout: rados osd op timeout = 900 2023-03-24T23:46:43.994 INFO:teuthology.orchestra.run.smithi072.stdout:[mon.a] 2023-03-24T23:46:43.994 INFO:teuthology.orchestra.run.smithi072.stdout:[mon.b] 2023-03-24T23:46:43.994 INFO:teuthology.orchestra.run.smithi072.stdout:[mon.c] 2023-03-24T23:46:43.994 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.a] 2023-03-24T23:46:43.995 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.c] 2023-03-24T23:46:43.995 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.b] 2023-03-24T23:46:43.995 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.d] 2023-03-24T23:46:43.995 INFO:teuthology.orchestra.run.smithi072.stdout:[mds] 2023-03-24T23:46:43.996 INFO:teuthology.orchestra.run.smithi072.stdout: debug mds = 20 2023-03-24T23:46:43.996 INFO:teuthology.orchestra.run.smithi072.stdout: debug mds balancer = 20 2023-03-24T23:46:43.996 INFO:teuthology.orchestra.run.smithi072.stdout: debug ms = 1 2023-03-24T23:46:43.996 INFO:teuthology.orchestra.run.smithi072.stdout: mds debug frag = True 2023-03-24T23:46:43.997 INFO:teuthology.orchestra.run.smithi072.stdout: mds debug scatterstat = True 2023-03-24T23:46:43.997 INFO:teuthology.orchestra.run.smithi072.stdout: mds op complaint time = 180 2023-03-24T23:46:43.997 INFO:teuthology.orchestra.run.smithi072.stdout: mds verify scatter = True 2023-03-24T23:46:43.998 INFO:teuthology.orchestra.run.smithi072.stdout: osd op complaint time = 180 2023-03-24T23:46:43.998 INFO:teuthology.orchestra.run.smithi072.stdout: rados mon op timeout = 900 2023-03-24T23:46:43.998 INFO:teuthology.orchestra.run.smithi072.stdout: rados osd op timeout = 900 2023-03-24T23:46:44.000 INFO:teuthology.orchestra.run.smithi146.stdout:[global] 2023-03-24T23:46:44.000 INFO:teuthology.orchestra.run.smithi146.stdout: chdir = "" 2023-03-24T23:46:44.000 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-03-24T23:46:44.000 INFO:teuthology.orchestra.run.smithi146.stdout: auth supported = cephx 2023-03-24T23:46:44.000 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:46:44.000 INFO:teuthology.orchestra.run.smithi146.stdout: filestore xattr use omap = true 2023-03-24T23:46:44.000 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:46:44.001 INFO:teuthology.orchestra.run.smithi146.stdout: mon clock drift allowed = 1.000 2023-03-24T23:46:44.001 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:46:44.001 INFO:teuthology.orchestra.run.smithi146.stdout: osd crush chooseleaf type = 0 2023-03-24T23:46:44.001 INFO:teuthology.orchestra.run.smithi146.stdout: auth debug = true 2023-03-24T23:46:44.001 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:46:44.001 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on old message = true 2023-03-24T23:46:44.001 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on bug = true 2023-03-24T23:46:44.002 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:46:44.002 INFO:teuthology.orchestra.run.smithi146.stdout: mon max pg per osd = 10000 # >= luminous 2023-03-24T23:46:44.002 INFO:teuthology.orchestra.run.smithi146.stdout: mon pg warn max object skew = 0 2023-03-24T23:46:44.002 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:46:44.002 INFO:teuthology.orchestra.run.smithi146.stdout: # disable pg_autoscaler by default for new pools 2023-03-24T23:46:44.002 INFO:teuthology.orchestra.run.smithi146.stdout: osd_pool_default_pg_autoscale_mode = off 2023-03-24T23:46:44.002 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:46:44.002 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default size = 2 2023-03-24T23:46:44.003 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:46:44.003 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow primary affinity = true 2023-03-24T23:46:44.003 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow pg remap = true 2023-03-24T23:46:44.003 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on legacy crush tunables = false 2023-03-24T23:46:44.003 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on crush straw calc version zero = false 2023-03-24T23:46:44.004 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on no sortbitwise = false 2023-03-24T23:46:44.004 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on osd down out interval zero = false 2023-03-24T23:46:44.004 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on too few osds = false 2023-03-24T23:46:44.004 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-03-24T23:46:44.004 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_no_redundancy = false 2023-03-24T23:46:44.004 INFO:teuthology.orchestra.run.smithi146.stdout: mon_allow_pool_size_one = true 2023-03-24T23:46:44.005 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:46:44.005 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-03-24T23:46:44.005 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:46:44.005 INFO:teuthology.orchestra.run.smithi146.stdout: osd default data pool replay window = 5 2023-03-24T23:46:44.006 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:46:44.006 INFO:teuthology.orchestra.run.smithi146.stdout: mon allow pool delete = true 2023-03-24T23:46:44.006 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:46:44.006 INFO:teuthology.orchestra.run.smithi146.stdout: mon cluster log file level = debug 2023-03-24T23:46:44.007 INFO:teuthology.orchestra.run.smithi146.stdout: debug asserts on shutdown = true 2023-03-24T23:46:44.007 INFO:teuthology.orchestra.run.smithi146.stdout: mon health detail to clog = false 2023-03-24T23:46:44.007 INFO:teuthology.orchestra.run.smithi146.stdout: mon host = "172.21.15.72,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]" 2023-03-24T23:46:44.008 INFO:teuthology.orchestra.run.smithi146.stdout: fsid = d3c122b7-f0b4-4c96-8f85-86f129e51880 2023-03-24T23:46:44.008 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:46:44.008 INFO:teuthology.orchestra.run.smithi146.stdout:[osd] 2023-03-24T23:46:44.008 INFO:teuthology.orchestra.run.smithi146.stdout: osd journal size = 100 2023-03-24T23:46:44.009 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:46:44.009 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub load threshold = 5.0 2023-03-24T23:46:44.009 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub max interval = 600 2023-03-24T23:46:44.009 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:46:44.009 INFO:teuthology.orchestra.run.smithi146.stdout: osd recover clone overlap = true 2023-03-24T23:46:44.009 INFO:teuthology.orchestra.run.smithi146.stdout: osd recovery max chunk = 1048576 2023-03-24T23:46:44.010 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:46:44.010 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug shutdown = true 2023-03-24T23:46:44.010 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug op order = false 2023-03-24T23:46:44.010 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug verify stray on activate = true 2023-03-24T23:46:44.011 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:46:44.011 INFO:teuthology.orchestra.run.smithi146.stdout: osd open classes on start = true 2023-03-24T23:46:44.011 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug pg log writeout = true 2023-03-24T23:46:44.011 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:46:44.011 INFO:teuthology.orchestra.run.smithi146.stdout: osd deep scrub update digest min age = 30 2023-03-24T23:46:44.011 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:46:44.011 INFO:teuthology.orchestra.run.smithi146.stdout: osd map max advance = 10 2023-03-24T23:46:44.011 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:46:44.012 INFO:teuthology.orchestra.run.smithi146.stdout: journal zero on create = true 2023-03-24T23:46:44.012 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:46:44.012 INFO:teuthology.orchestra.run.smithi146.stdout: filestore ondisk finisher threads = 3 2023-03-24T23:46:44.012 INFO:teuthology.orchestra.run.smithi146.stdout: filestore apply finisher threads = 3 2023-03-24T23:46:44.012 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:46:44.012 INFO:teuthology.orchestra.run.smithi146.stdout: bdev debug aio = true 2023-03-24T23:46:44.012 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug misdirected ops = true 2023-03-24T23:46:44.012 INFO:teuthology.orchestra.run.smithi146.stdout: bdev async discard = True 2023-03-24T23:46:44.013 INFO:teuthology.orchestra.run.smithi146.stdout: bdev enable discard = True 2023-03-24T23:46:44.013 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore allocator = bitmap 2023-03-24T23:46:44.013 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore block size = 96636764160 2023-03-24T23:46:44.013 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore fsck on mount = True 2023-03-24T23:46:44.013 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluefs = 1/20 2023-03-24T23:46:44.013 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluestore = 1/20 2023-03-24T23:46:44.013 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-03-24T23:46:44.014 INFO:teuthology.orchestra.run.smithi146.stdout: debug osd = 20 2023-03-24T23:46:44.014 INFO:teuthology.orchestra.run.smithi146.stdout: debug rocksdb = 4/10 2023-03-24T23:46:44.014 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd backfillfull_ratio = 0.85 2023-03-24T23:46:44.014 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd full ratio = 0.9 2023-03-24T23:46:44.014 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd nearfull ratio = 0.8 2023-03-24T23:46:44.014 INFO:teuthology.orchestra.run.smithi146.stdout: osd failsafe full ratio = 0.95 2023-03-24T23:46:44.014 INFO:teuthology.orchestra.run.smithi146.stdout: osd objectstore = bluestore 2023-03-24T23:46:44.014 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2023-03-24T23:46:44.015 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:46:44.015 INFO:teuthology.orchestra.run.smithi146.stdout:[mgr] 2023-03-24T23:46:44.015 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-03-24T23:46:44.015 INFO:teuthology.orchestra.run.smithi146.stdout: debug mgr = 20 2023-03-24T23:46:44.015 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2023-03-24T23:46:44.015 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2023-03-24T23:46:44.015 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2023-03-24T23:46:44.015 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2023-03-24T23:46:44.016 INFO:teuthology.orchestra.run.smithi146.stdout: mgr/telemetry/nag = false 2023-03-24T23:46:44.016 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:46:44.016 INFO:teuthology.orchestra.run.smithi146.stdout:[mon] 2023-03-24T23:46:44.016 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-03-24T23:46:44.016 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2023-03-24T23:46:44.016 INFO:teuthology.orchestra.run.smithi146.stdout: debug paxos = 20 2023-03-24T23:46:44.016 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2023-03-24T23:46:44.016 INFO:teuthology.orchestra.run.smithi146.stdout: mon data avail warn = 5 2023-03-24T23:46:44.017 INFO:teuthology.orchestra.run.smithi146.stdout: mon mgr mkfs grace = 240 2023-03-24T23:46:44.017 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2023-03-24T23:46:44.017 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd reporter subtree level = osd 2023-03-24T23:46:44.017 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd prime pg temp = true 2023-03-24T23:46:44.017 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2023-03-24T23:46:44.017 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:46:44.017 INFO:teuthology.orchestra.run.smithi146.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-03-24T23:46:44.018 INFO:teuthology.orchestra.run.smithi146.stdout: auth mon ticket ttl = 660 # 11m 2023-03-24T23:46:44.018 INFO:teuthology.orchestra.run.smithi146.stdout: auth service ticket ttl = 240 # 4m 2023-03-24T23:46:44.018 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:46:44.018 INFO:teuthology.orchestra.run.smithi146.stdout: # don't complain about insecure global_id in the test suite 2023-03-24T23:46:44.018 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-03-24T23:46:44.018 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-03-24T23:46:44.018 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:46:44.018 INFO:teuthology.orchestra.run.smithi146.stdout: # 1m isn't quite enough 2023-03-24T23:46:44.019 INFO:teuthology.orchestra.run.smithi146.stdout: mon_down_mkfs_grace = 2m 2023-03-24T23:46:44.019 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:46:44.019 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_filestore_osds = false 2023-03-24T23:46:44.019 INFO:teuthology.orchestra.run.smithi146.stdout: mon op complaint time = 120 2023-03-24T23:46:44.019 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-03-24T23:46:44.019 INFO:teuthology.orchestra.run.smithi146.stdout:[client] 2023-03-24T23:46:44.020 INFO:teuthology.orchestra.run.smithi146.stdout: rgw cache enabled = true 2023-03-24T23:46:44.020 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable ops log = true 2023-03-24T23:46:44.020 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable usage log = true 2023-03-24T23:46:44.020 INFO:teuthology.orchestra.run.smithi146.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-03-24T23:46:44.021 INFO:teuthology.orchestra.run.smithi146.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-03-24T23:46:44.021 INFO:teuthology.orchestra.run.smithi146.stdout: client mount timeout = 600 2023-03-24T23:46:44.021 INFO:teuthology.orchestra.run.smithi146.stdout: debug client = 20 2023-03-24T23:46:44.021 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-03-24T23:46:44.022 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = "" 2023-03-24T23:46:44.022 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2023-03-24T23:46:44.022 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2023-03-24T23:46:44.022 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.a] 2023-03-24T23:46:44.023 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.b] 2023-03-24T23:46:44.023 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.c] 2023-03-24T23:46:44.023 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.a] 2023-03-24T23:46:44.023 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.c] 2023-03-24T23:46:44.024 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.b] 2023-03-24T23:46:44.024 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.d] 2023-03-24T23:46:44.024 INFO:teuthology.orchestra.run.smithi146.stdout:[mds] 2023-03-24T23:46:44.024 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds = 20 2023-03-24T23:46:44.025 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds balancer = 20 2023-03-24T23:46:44.025 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-03-24T23:46:44.025 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug frag = True 2023-03-24T23:46:44.026 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug scatterstat = True 2023-03-24T23:46:44.026 INFO:teuthology.orchestra.run.smithi146.stdout: mds op complaint time = 180 2023-03-24T23:46:44.026 INFO:teuthology.orchestra.run.smithi146.stdout: mds verify scatter = True 2023-03-24T23:46:44.026 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2023-03-24T23:46:44.026 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2023-03-24T23:46:44.026 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2023-03-24T23:46:44.027 INFO:tasks.ceph:Creating admin key on mon.a... 2023-03-24T23:46:44.027 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-03-24T23:46:44.097 INFO:tasks.ceph:Copying monmap to all nodes... 2023-03-24T23:46:44.097 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-24T23:46:44.097 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-03-24T23:46:44.115 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-24T23:46:44.115 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-03-24T23:46:44.172 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi072.front.sepia.ceph.com 2023-03-24T23:46:44.172 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-24T23:46:44.172 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/ceph/ceph.keyring 2023-03-24T23:46:44.173 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-03-24T23:46:44.248 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-24T23:46:44.248 DEBUG:teuthology.orchestra.run.smithi072:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-03-24T23:46:44.305 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi146.front.sepia.ceph.com 2023-03-24T23:46:44.306 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-03-24T23:46:44.306 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/ceph/ceph.keyring 2023-03-24T23:46:44.306 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-03-24T23:46:44.345 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-03-24T23:46:44.346 DEBUG:teuthology.orchestra.run.smithi146:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-03-24T23:46:44.404 INFO:tasks.ceph:Setting up mon nodes... 2023-03-24T23:46:44.404 INFO:tasks.ceph:Setting up mgr nodes... 2023-03-24T23:46:44.405 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-03-24T23:46:44.458 INFO:teuthology.orchestra.run.smithi072.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-03-24T23:46:44.461 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-03-24T23:46:44.531 INFO:teuthology.orchestra.run.smithi146.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-03-24T23:46:44.534 INFO:tasks.ceph:Setting up mds nodes... 2023-03-24T23:46:44.534 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-03-24T23:46:44.588 INFO:teuthology.orchestra.run.smithi072.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-03-24T23:46:44.590 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-03-24T23:46:44.618 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-03-24T23:46:44.711 INFO:teuthology.orchestra.run.smithi072.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-03-24T23:46:44.713 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-03-24T23:46:44.740 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-03-24T23:46:44.794 INFO:teuthology.orchestra.run.smithi146.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-03-24T23:46:44.797 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-03-24T23:46:44.823 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-03-24T23:46:44.916 INFO:teuthology.orchestra.run.smithi146.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-03-24T23:46:44.918 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-03-24T23:46:44.945 INFO:tasks.ceph_client:Setting up client nodes... 2023-03-24T23:46:44.946 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-03-24T23:46:44.989 INFO:teuthology.orchestra.run.smithi072.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-03-24T23:46:45.002 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-03-24T23:46:45.085 INFO:teuthology.orchestra.run.smithi072.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-03-24T23:46:45.098 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-03-24T23:46:45.144 INFO:teuthology.orchestra.run.smithi146.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-03-24T23:46:45.157 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-03-24T23:46:45.201 INFO:teuthology.orchestra.run.smithi146.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-03-24T23:46:45.214 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-03-24T23:46:45.215 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi072.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi146.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-03-24T23:46:45.215 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-03-24T23:46:45.242 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-03-24T23:46:45.242 INFO:tasks.ceph:role: osd.0 2023-03-24T23:46:45.242 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi072.front.sepia.ceph.com 2023-03-24T23:46:45.243 DEBUG:teuthology.orchestra.run.smithi072:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-03-24T23:46:45.316 INFO:teuthology.orchestra.run.smithi072.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-03-24T23:46:45.317 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-24T23:46:45.317 INFO:teuthology.orchestra.run.smithi072.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-24T23:46:45.317 INFO:teuthology.orchestra.run.smithi072.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-24T23:46:45.317 INFO:teuthology.orchestra.run.smithi072.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-24T23:46:45.317 INFO:teuthology.orchestra.run.smithi072.stdout: = sunit=0 swidth=0 blks 2023-03-24T23:46:45.317 INFO:teuthology.orchestra.run.smithi072.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-24T23:46:45.318 INFO:teuthology.orchestra.run.smithi072.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-24T23:46:45.318 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-24T23:46:45.318 INFO:teuthology.orchestra.run.smithi072.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-24T23:46:45.744 INFO:teuthology.orchestra.run.smithi072.stdout:Discarding blocks...Done. 2023-03-24T23:46:45.746 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi072.front.sepia.ceph.com -o noatime 2023-03-24T23:46:45.746 DEBUG:teuthology.orchestra.run.smithi072:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-03-24T23:46:45.905 DEBUG:teuthology.orchestra.run.smithi072:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-03-24T23:46:45.942 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-03-24T23:46:46.012 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-03-24T23:46:46.012 INFO:tasks.ceph:role: osd.1 2023-03-24T23:46:46.012 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi072.front.sepia.ceph.com 2023-03-24T23:46:46.013 DEBUG:teuthology.orchestra.run.smithi072:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-03-24T23:46:46.087 INFO:teuthology.orchestra.run.smithi072.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-03-24T23:46:46.088 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-24T23:46:46.088 INFO:teuthology.orchestra.run.smithi072.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-24T23:46:46.088 INFO:teuthology.orchestra.run.smithi072.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-24T23:46:46.088 INFO:teuthology.orchestra.run.smithi072.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-24T23:46:46.088 INFO:teuthology.orchestra.run.smithi072.stdout: = sunit=0 swidth=0 blks 2023-03-24T23:46:46.088 INFO:teuthology.orchestra.run.smithi072.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-24T23:46:46.088 INFO:teuthology.orchestra.run.smithi072.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-24T23:46:46.088 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-24T23:46:46.089 INFO:teuthology.orchestra.run.smithi072.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-24T23:46:46.521 INFO:teuthology.orchestra.run.smithi072.stdout:Discarding blocks...Done. 2023-03-24T23:46:46.522 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi072.front.sepia.ceph.com -o noatime 2023-03-24T23:46:46.523 DEBUG:teuthology.orchestra.run.smithi072:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-03-24T23:46:46.629 DEBUG:teuthology.orchestra.run.smithi072:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-03-24T23:46:46.661 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-03-24T23:46:46.728 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-03-24T23:46:46.728 INFO:tasks.ceph:role: osd.2 2023-03-24T23:46:46.729 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi072.front.sepia.ceph.com 2023-03-24T23:46:46.729 DEBUG:teuthology.orchestra.run.smithi072:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-03-24T23:46:46.803 INFO:teuthology.orchestra.run.smithi072.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-03-24T23:46:46.803 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-24T23:46:46.803 INFO:teuthology.orchestra.run.smithi072.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-24T23:46:46.803 INFO:teuthology.orchestra.run.smithi072.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-24T23:46:46.803 INFO:teuthology.orchestra.run.smithi072.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-24T23:46:46.803 INFO:teuthology.orchestra.run.smithi072.stdout: = sunit=0 swidth=0 blks 2023-03-24T23:46:46.804 INFO:teuthology.orchestra.run.smithi072.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-24T23:46:46.804 INFO:teuthology.orchestra.run.smithi072.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-24T23:46:46.804 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-24T23:46:46.804 INFO:teuthology.orchestra.run.smithi072.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-24T23:46:47.254 INFO:teuthology.orchestra.run.smithi072.stdout:Discarding blocks...Done. 2023-03-24T23:46:47.255 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi072.front.sepia.ceph.com -o noatime 2023-03-24T23:46:47.255 DEBUG:teuthology.orchestra.run.smithi072:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-03-24T23:46:47.447 DEBUG:teuthology.orchestra.run.smithi072:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-03-24T23:46:47.479 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-03-24T23:46:47.547 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-03-24T23:46:47.548 INFO:tasks.ceph:role: osd.3 2023-03-24T23:46:47.548 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi072.front.sepia.ceph.com 2023-03-24T23:46:47.548 DEBUG:teuthology.orchestra.run.smithi072:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-03-24T23:46:47.622 INFO:teuthology.orchestra.run.smithi072.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-03-24T23:46:47.623 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-24T23:46:47.623 INFO:teuthology.orchestra.run.smithi072.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-24T23:46:47.623 INFO:teuthology.orchestra.run.smithi072.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-24T23:46:47.623 INFO:teuthology.orchestra.run.smithi072.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-24T23:46:47.623 INFO:teuthology.orchestra.run.smithi072.stdout: = sunit=0 swidth=0 blks 2023-03-24T23:46:47.623 INFO:teuthology.orchestra.run.smithi072.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-24T23:46:47.624 INFO:teuthology.orchestra.run.smithi072.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-24T23:46:47.624 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-24T23:46:47.624 INFO:teuthology.orchestra.run.smithi072.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-24T23:46:48.045 INFO:teuthology.orchestra.run.smithi072.stdout:Discarding blocks...Done. 2023-03-24T23:46:48.047 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi072.front.sepia.ceph.com -o noatime 2023-03-24T23:46:48.047 DEBUG:teuthology.orchestra.run.smithi072:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-03-24T23:46:48.245 DEBUG:teuthology.orchestra.run.smithi072:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-03-24T23:46:48.277 DEBUG:teuthology.orchestra.run.smithi072:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-24T23:46:48.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:46:48.378+0000 7f0cdfb90640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-03-24T23:46:48.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:46:48.378+0000 7f0cdfb90640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-03-24T23:46:48.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:46:48.378+0000 7f0cdfb90640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-03-24T23:46:48.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:46:48.378+0000 7f0cdfb90640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-03-24T23:46:48.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:46:48.378+0000 7f0cdfb90640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-03-24T23:46:48.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:46:48.379+0000 7f0cdfb90640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-03-24T23:46:51.228 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-03-24T23:46:51.258 DEBUG:teuthology.orchestra.run.smithi072:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-24T23:46:51.349 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:46:51.347+0000 7f0a15b57640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-03-24T23:46:51.349 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:46:51.347+0000 7f0a15b57640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-03-24T23:46:51.349 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:46:51.347+0000 7f0a15b57640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-03-24T23:46:51.349 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:46:51.347+0000 7f0a15b57640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-03-24T23:46:51.349 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:46:51.347+0000 7f0a15b57640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-03-24T23:46:51.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:46:51.348+0000 7f0a15b57640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-03-24T23:46:54.200 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-03-24T23:46:54.232 DEBUG:teuthology.orchestra.run.smithi072:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-24T23:46:54.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:46:54.323+0000 7f5d22c49640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-03-24T23:46:54.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:46:54.324+0000 7f5d22c49640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-03-24T23:46:54.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:46:54.324+0000 7f5d22c49640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-03-24T23:46:54.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:46:54.324+0000 7f5d22c49640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-03-24T23:46:54.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:46:54.324+0000 7f5d22c49640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-03-24T23:46:54.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:46:54.324+0000 7f5d22c49640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-03-24T23:46:57.184 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-03-24T23:46:57.215 DEBUG:teuthology.orchestra.run.smithi072:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-24T23:46:57.455 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:46:57.454+0000 7f44d7555640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-03-24T23:46:57.455 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:46:57.454+0000 7f44d7555640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-03-24T23:46:57.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:46:57.454+0000 7f44d7555640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-03-24T23:46:57.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:46:57.454+0000 7f44d7555640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-03-24T23:46:57.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:46:57.454+0000 7f44d7555640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-03-24T23:46:57.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:46:57.455+0000 7f44d7555640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-03-24T23:47:00.307 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-03-24T23:47:00.336 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-03-24T23:47:00.367 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-03-24T23:47:00.367 INFO:tasks.ceph:role: osd.4 2023-03-24T23:47:00.368 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi146.front.sepia.ceph.com 2023-03-24T23:47:00.368 DEBUG:teuthology.orchestra.run.smithi146:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-03-24T23:47:00.443 INFO:teuthology.orchestra.run.smithi146.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-03-24T23:47:00.444 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-24T23:47:00.444 INFO:teuthology.orchestra.run.smithi146.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-24T23:47:00.444 INFO:teuthology.orchestra.run.smithi146.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-24T23:47:00.444 INFO:teuthology.orchestra.run.smithi146.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-24T23:47:00.445 INFO:teuthology.orchestra.run.smithi146.stdout: = sunit=0 swidth=0 blks 2023-03-24T23:47:00.445 INFO:teuthology.orchestra.run.smithi146.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-24T23:47:00.445 INFO:teuthology.orchestra.run.smithi146.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-24T23:47:00.445 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-24T23:47:00.446 INFO:teuthology.orchestra.run.smithi146.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-24T23:47:00.867 INFO:teuthology.orchestra.run.smithi146.stdout:Discarding blocks...Done. 2023-03-24T23:47:00.869 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi146.front.sepia.ceph.com -o noatime 2023-03-24T23:47:00.870 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-03-24T23:47:01.054 DEBUG:teuthology.orchestra.run.smithi146:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-03-24T23:47:01.090 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-03-24T23:47:01.159 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-03-24T23:47:01.159 INFO:tasks.ceph:role: osd.5 2023-03-24T23:47:01.159 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi146.front.sepia.ceph.com 2023-03-24T23:47:01.160 DEBUG:teuthology.orchestra.run.smithi146:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-03-24T23:47:01.234 INFO:teuthology.orchestra.run.smithi146.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-03-24T23:47:01.234 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-24T23:47:01.235 INFO:teuthology.orchestra.run.smithi146.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-24T23:47:01.235 INFO:teuthology.orchestra.run.smithi146.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-24T23:47:01.235 INFO:teuthology.orchestra.run.smithi146.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-24T23:47:01.235 INFO:teuthology.orchestra.run.smithi146.stdout: = sunit=0 swidth=0 blks 2023-03-24T23:47:01.235 INFO:teuthology.orchestra.run.smithi146.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-24T23:47:01.235 INFO:teuthology.orchestra.run.smithi146.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-24T23:47:01.236 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-24T23:47:01.236 INFO:teuthology.orchestra.run.smithi146.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-24T23:47:01.657 INFO:teuthology.orchestra.run.smithi146.stdout:Discarding blocks...Done. 2023-03-24T23:47:01.658 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi146.front.sepia.ceph.com -o noatime 2023-03-24T23:47:01.659 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-03-24T23:47:01.767 DEBUG:teuthology.orchestra.run.smithi146:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-03-24T23:47:01.800 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-03-24T23:47:01.868 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-03-24T23:47:01.868 INFO:tasks.ceph:role: osd.6 2023-03-24T23:47:01.869 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi146.front.sepia.ceph.com 2023-03-24T23:47:01.869 DEBUG:teuthology.orchestra.run.smithi146:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-03-24T23:47:01.942 INFO:teuthology.orchestra.run.smithi146.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-03-24T23:47:01.943 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-24T23:47:01.943 INFO:teuthology.orchestra.run.smithi146.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-24T23:47:01.943 INFO:teuthology.orchestra.run.smithi146.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-24T23:47:01.943 INFO:teuthology.orchestra.run.smithi146.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-24T23:47:01.943 INFO:teuthology.orchestra.run.smithi146.stdout: = sunit=0 swidth=0 blks 2023-03-24T23:47:01.943 INFO:teuthology.orchestra.run.smithi146.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-24T23:47:01.943 INFO:teuthology.orchestra.run.smithi146.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-24T23:47:01.944 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-24T23:47:01.944 INFO:teuthology.orchestra.run.smithi146.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-24T23:47:02.390 INFO:teuthology.orchestra.run.smithi146.stdout:Discarding blocks...Done. 2023-03-24T23:47:02.391 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi146.front.sepia.ceph.com -o noatime 2023-03-24T23:47:02.392 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-03-24T23:47:02.573 DEBUG:teuthology.orchestra.run.smithi146:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-03-24T23:47:02.606 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-03-24T23:47:02.675 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-03-24T23:47:02.676 INFO:tasks.ceph:role: osd.7 2023-03-24T23:47:02.676 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi146.front.sepia.ceph.com 2023-03-24T23:47:02.676 DEBUG:teuthology.orchestra.run.smithi146:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-03-24T23:47:02.751 INFO:teuthology.orchestra.run.smithi146.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-03-24T23:47:02.752 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-24T23:47:02.752 INFO:teuthology.orchestra.run.smithi146.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-24T23:47:02.752 INFO:teuthology.orchestra.run.smithi146.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-24T23:47:02.753 INFO:teuthology.orchestra.run.smithi146.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-24T23:47:02.753 INFO:teuthology.orchestra.run.smithi146.stdout: = sunit=0 swidth=0 blks 2023-03-24T23:47:02.753 INFO:teuthology.orchestra.run.smithi146.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-24T23:47:02.753 INFO:teuthology.orchestra.run.smithi146.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-24T23:47:02.754 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-24T23:47:02.754 INFO:teuthology.orchestra.run.smithi146.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-24T23:47:03.195 INFO:teuthology.orchestra.run.smithi146.stdout:Discarding blocks...Done. 2023-03-24T23:47:03.196 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi146.front.sepia.ceph.com -o noatime 2023-03-24T23:47:03.197 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-03-24T23:47:03.385 DEBUG:teuthology.orchestra.run.smithi146:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-03-24T23:47:03.419 DEBUG:teuthology.orchestra.run.smithi146:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-24T23:47:03.517 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:03.517+0000 7ff212f17640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-03-24T23:47:03.518 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:03.517+0000 7ff212f17640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-03-24T23:47:03.518 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:03.517+0000 7ff212f17640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-03-24T23:47:03.518 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:03.517+0000 7ff212f17640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-03-24T23:47:03.518 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:03.517+0000 7ff212f17640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-03-24T23:47:03.518 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:03.518+0000 7ff212f17640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-03-24T23:47:06.364 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-03-24T23:47:06.396 DEBUG:teuthology.orchestra.run.smithi146:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-24T23:47:06.484 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:06.484+0000 7f7756d51640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-03-24T23:47:06.484 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:06.484+0000 7f7756d51640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-03-24T23:47:06.485 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:06.484+0000 7f7756d51640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-03-24T23:47:06.485 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:06.484+0000 7f7756d51640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-03-24T23:47:06.485 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:06.484+0000 7f7756d51640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-03-24T23:47:06.485 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:06.485+0000 7f7756d51640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-03-24T23:47:09.334 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-03-24T23:47:09.366 DEBUG:teuthology.orchestra.run.smithi146:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-24T23:47:09.457 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:09.457+0000 7f790bb64640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-03-24T23:47:09.458 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:09.457+0000 7f790bb64640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-03-24T23:47:09.458 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:09.457+0000 7f790bb64640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-03-24T23:47:09.458 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:09.457+0000 7f790bb64640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-03-24T23:47:09.458 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:09.457+0000 7f790bb64640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-03-24T23:47:09.458 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:09.458+0000 7f790bb64640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-03-24T23:47:11.938 DEBUG:teuthology.orchestra.run.smithi072:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-24T23:47:11.941 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-24T23:47:12.310 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-03-24T23:47:12.337 DEBUG:teuthology.orchestra.run.smithi146:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-24T23:47:12.425 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:12.424+0000 7ff95c936640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-03-24T23:47:12.425 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:12.425+0000 7ff95c936640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-03-24T23:47:12.425 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:12.425+0000 7ff95c936640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-03-24T23:47:12.425 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:12.425+0000 7ff95c936640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-03-24T23:47:12.426 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:12.425+0000 7ff95c936640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-03-24T23:47:12.426 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:12.425+0000 7ff95c936640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-03-24T23:47:15.267 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-03-24T23:47:15.295 INFO:tasks.ceph:Reading keys from all nodes... 2023-03-24T23:47:15.296 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-24T23:47:15.296 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-03-24T23:47:15.324 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-24T23:47:15.325 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-03-24T23:47:15.392 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-24T23:47:15.393 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-03-24T23:47:15.459 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-24T23:47:15.460 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-03-24T23:47:15.525 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-24T23:47:15.526 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-03-24T23:47:15.593 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-24T23:47:15.594 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-03-24T23:47:15.661 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-24T23:47:15.661 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-03-24T23:47:15.727 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-03-24T23:47:15.728 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-03-24T23:47:15.755 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-03-24T23:47:15.755 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-03-24T23:47:15.822 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-03-24T23:47:15.823 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-03-24T23:47:15.888 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-03-24T23:47:15.889 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-03-24T23:47:15.955 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-03-24T23:47:15.956 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-03-24T23:47:16.023 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-03-24T23:47:16.024 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-03-24T23:47:16.090 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-03-24T23:47:16.091 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-03-24T23:47:16.158 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-24T23:47:16.158 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-03-24T23:47:16.176 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-24T23:47:16.176 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-03-24T23:47:16.233 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-03-24T23:47:16.234 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-03-24T23:47:16.252 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-03-24T23:47:16.252 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-03-24T23:47:16.309 INFO:tasks.ceph:Adding keys to all mons... 2023-03-24T23:47:16.310 DEBUG:teuthology.orchestra.run.smithi072:> sudo tee -a /etc/ceph/ceph.keyring 2023-03-24T23:47:16.312 DEBUG:teuthology.orchestra.run.smithi146:> sudo tee -a /etc/ceph/ceph.keyring 2023-03-24T23:47:16.353 INFO:teuthology.orchestra.run.smithi072.stdout:[mgr.y] 2023-03-24T23:47:16.353 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQBkNh5kCDBaGxAAp3K2DYUHZoFIPVa8jOhk4w== 2023-03-24T23:47:16.353 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.a] 2023-03-24T23:47:16.354 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQBkNh5k/dgYIxAAz1oDNV3+Nt4cqURB+HL9pQ== 2023-03-24T23:47:16.354 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.c] 2023-03-24T23:47:16.354 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQBkNh5kd9JtKhAA+XlkSYt5lGqnhAdkaWNyTw== 2023-03-24T23:47:16.354 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.0] 2023-03-24T23:47:16.354 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQBoNh5k3I+mFhAAReQm2wYSIp+uDmJItFMSSw== 2023-03-24T23:47:16.355 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.1] 2023-03-24T23:47:16.355 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQBrNh5kGv3MFBAAJ4hI7Kl2mKsWfk2ZyEAYmA== 2023-03-24T23:47:16.355 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.2] 2023-03-24T23:47:16.355 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQBuNh5kGENhExAAZLLUb3bOGc8bncPrdxC5dQ== 2023-03-24T23:47:16.355 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.3] 2023-03-24T23:47:16.355 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQBxNh5kN2knGxAAS0WvWH7qbHgz4lnwW7fCMw== 2023-03-24T23:47:16.356 INFO:teuthology.orchestra.run.smithi072.stdout:[mgr.x] 2023-03-24T23:47:16.356 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQBkNh5kDpCyHxAAxLZJfHVVM9Tb4kz5xF7XJw== 2023-03-24T23:47:16.356 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.b] 2023-03-24T23:47:16.357 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQBkNh5kfr5kLxAAs8E9HEuumHnI9wXYawQpFg== 2023-03-24T23:47:16.357 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.d] 2023-03-24T23:47:16.357 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQBkNh5kA9WnNhAASJuMBmAV+FDhFkSc4C0aZQ== 2023-03-24T23:47:16.357 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.4] 2023-03-24T23:47:16.358 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQB3Nh5keiLdHhAAQtRT1EThf/JblO3H9QGLBg== 2023-03-24T23:47:16.358 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.5] 2023-03-24T23:47:16.358 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQB6Nh5kP/3hHBAAs9K7MsXItOgmE/6Qen2sXw== 2023-03-24T23:47:16.358 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.6] 2023-03-24T23:47:16.359 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQB9Nh5k3ztJGxAApJazFANqwKr2vQW5qSk7RQ== 2023-03-24T23:47:16.359 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.7] 2023-03-24T23:47:16.359 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQCANh5kCMxXGRAAupSVBoE8WUDTFFqKaXYpDQ== 2023-03-24T23:47:16.359 INFO:teuthology.orchestra.run.smithi072.stdout:[client.0] 2023-03-24T23:47:16.359 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQBkNh5k9r4FOxAAfIKYR9TekOBPW/QZ8xYLtQ== 2023-03-24T23:47:16.359 INFO:teuthology.orchestra.run.smithi072.stdout:[client.2] 2023-03-24T23:47:16.360 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQBlNh5kubohBRAAO9ymEAcZytUIX5C+6Y9ERg== 2023-03-24T23:47:16.360 INFO:teuthology.orchestra.run.smithi072.stdout:[client.1] 2023-03-24T23:47:16.360 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQBlNh5kNOWnCBAAQtadmbdlUt1c3JLaNUSBCA== 2023-03-24T23:47:16.360 INFO:teuthology.orchestra.run.smithi072.stdout:[client.3] 2023-03-24T23:47:16.360 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQBlNh5kD7EODBAAxZZcQemOsw8Pmx3NBwMylw== 2023-03-24T23:47:16.375 INFO:teuthology.orchestra.run.smithi146.stdout:[mgr.y] 2023-03-24T23:47:16.376 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQBkNh5kCDBaGxAAp3K2DYUHZoFIPVa8jOhk4w== 2023-03-24T23:47:16.376 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.a] 2023-03-24T23:47:16.376 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQBkNh5k/dgYIxAAz1oDNV3+Nt4cqURB+HL9pQ== 2023-03-24T23:47:16.376 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.c] 2023-03-24T23:47:16.376 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQBkNh5kd9JtKhAA+XlkSYt5lGqnhAdkaWNyTw== 2023-03-24T23:47:16.376 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.0] 2023-03-24T23:47:16.377 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQBoNh5k3I+mFhAAReQm2wYSIp+uDmJItFMSSw== 2023-03-24T23:47:16.377 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.1] 2023-03-24T23:47:16.377 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQBrNh5kGv3MFBAAJ4hI7Kl2mKsWfk2ZyEAYmA== 2023-03-24T23:47:16.377 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.2] 2023-03-24T23:47:16.377 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQBuNh5kGENhExAAZLLUb3bOGc8bncPrdxC5dQ== 2023-03-24T23:47:16.377 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.3] 2023-03-24T23:47:16.378 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQBxNh5kN2knGxAAS0WvWH7qbHgz4lnwW7fCMw== 2023-03-24T23:47:16.378 INFO:teuthology.orchestra.run.smithi146.stdout:[mgr.x] 2023-03-24T23:47:16.378 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQBkNh5kDpCyHxAAxLZJfHVVM9Tb4kz5xF7XJw== 2023-03-24T23:47:16.378 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.b] 2023-03-24T23:47:16.378 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQBkNh5kfr5kLxAAs8E9HEuumHnI9wXYawQpFg== 2023-03-24T23:47:16.378 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.d] 2023-03-24T23:47:16.378 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQBkNh5kA9WnNhAASJuMBmAV+FDhFkSc4C0aZQ== 2023-03-24T23:47:16.379 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.4] 2023-03-24T23:47:16.379 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQB3Nh5keiLdHhAAQtRT1EThf/JblO3H9QGLBg== 2023-03-24T23:47:16.379 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.5] 2023-03-24T23:47:16.379 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQB6Nh5kP/3hHBAAs9K7MsXItOgmE/6Qen2sXw== 2023-03-24T23:47:16.379 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.6] 2023-03-24T23:47:16.379 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQB9Nh5k3ztJGxAApJazFANqwKr2vQW5qSk7RQ== 2023-03-24T23:47:16.379 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.7] 2023-03-24T23:47:16.380 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCANh5kCMxXGRAAupSVBoE8WUDTFFqKaXYpDQ== 2023-03-24T23:47:16.380 INFO:teuthology.orchestra.run.smithi146.stdout:[client.0] 2023-03-24T23:47:16.380 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQBkNh5k9r4FOxAAfIKYR9TekOBPW/QZ8xYLtQ== 2023-03-24T23:47:16.380 INFO:teuthology.orchestra.run.smithi146.stdout:[client.2] 2023-03-24T23:47:16.380 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQBlNh5kubohBRAAO9ymEAcZytUIX5C+6Y9ERg== 2023-03-24T23:47:16.380 INFO:teuthology.orchestra.run.smithi146.stdout:[client.1] 2023-03-24T23:47:16.380 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQBlNh5kNOWnCBAAQtadmbdlUt1c3JLaNUSBCA== 2023-03-24T23:47:16.381 INFO:teuthology.orchestra.run.smithi146.stdout:[client.3] 2023-03-24T23:47:16.381 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQBlNh5kD7EODBAAxZZcQemOsw8Pmx3NBwMylw== 2023-03-24T23:47:16.382 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-03-24T23:47:16.404 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-03-24T23:47:16.475 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-24T23:47:16.478 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-24T23:47:16.534 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-24T23:47:16.536 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-24T23:47:16.593 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-24T23:47:16.595 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-24T23:47:16.648 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-24T23:47:16.651 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-24T23:47:16.704 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-24T23:47:16.706 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-24T23:47:16.759 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-24T23:47:16.761 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-24T23:47:16.815 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-03-24T23:47:16.817 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-03-24T23:47:16.871 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-24T23:47:16.873 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-24T23:47:16.929 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-24T23:47:16.932 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-24T23:47:16.989 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-24T23:47:16.991 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-24T23:47:17.044 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-24T23:47:17.047 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-24T23:47:17.100 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-24T23:47:17.102 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-24T23:47:17.156 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-24T23:47:17.159 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-24T23:47:17.212 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-24T23:47:17.215 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-24T23:47:17.272 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-24T23:47:17.274 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-24T23:47:17.330 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-24T23:47:17.332 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-24T23:47:17.390 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-24T23:47:17.392 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-24T23:47:17.450 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-03-24T23:47:17.450 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-03-24T23:47:17.477 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-03-24T23:47:17.576 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-03-24T23:47:17.603 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-03-24T23:47:17.631 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-03-24T23:47:17.731 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-03-24T23:47:17.758 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-03-24T23:47:17.825 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-03-24T23:47:17.926 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-03-24T23:47:17.953 DEBUG:teuthology.orchestra.run.smithi072:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-03-24T23:47:17.956 DEBUG:teuthology.orchestra.run.smithi146:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-03-24T23:47:18.011 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-03-24T23:47:18.012 INFO:tasks.ceph.mon.a:Restarting daemon 2023-03-24T23:47:18.012 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-03-24T23:47:18.015 INFO:tasks.ceph.mon.a:Started 2023-03-24T23:47:18.015 INFO:tasks.ceph.mon.b:Restarting daemon 2023-03-24T23:47:18.016 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-03-24T23:47:18.054 INFO:tasks.ceph.mon.b:Started 2023-03-24T23:47:18.054 INFO:tasks.ceph.mon.c:Restarting daemon 2023-03-24T23:47:18.055 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-03-24T23:47:18.057 INFO:tasks.ceph.mon.c:Started 2023-03-24T23:47:18.057 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-03-24T23:47:18.057 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-03-24T23:47:18.058 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-03-24T23:47:18.060 INFO:tasks.ceph.mgr.y:Started 2023-03-24T23:47:18.060 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-03-24T23:47:18.060 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-03-24T23:47:18.062 INFO:tasks.ceph.mgr.x:Started 2023-03-24T23:47:18.062 INFO:tasks.ceph:Setting crush tunables to default 2023-03-24T23:47:18.063 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph --cluster ceph osd crush tunables default 2023-03-24T23:47:18.185 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.185+0000 7faa1663c700 1 Processor -- start 2023-03-24T23:47:18.185 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.185+0000 7faa1663c700 1 -- start start 2023-03-24T23:47:18.187 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.185+0000 7faa1663c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa1015cd30 0x7faa100d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:18.187 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.185+0000 7faa1663c700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7faa10071890 0x7faa100b2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:18.187 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.185+0000 7faa1663c700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa10071220 con 0x7faa100c9f20 2023-03-24T23:47:18.188 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.185+0000 7faa1663c700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa1006be90 con 0x7faa10071890 2023-03-24T23:47:18.188 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.185+0000 7faa1663c700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa10071470 con 0x7faa1015cd30 2023-03-24T23:47:18.188 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.186+0000 7faa0f7fe700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7faa10071890 0x7faa100b2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:18.188 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.186+0000 7faa14bd9700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7faa100c9f20 0x7faa10160bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:40096/0 (socket says 172.21.15.72:40096) 2023-03-24T23:47:18.188 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.186+0000 7faa0f7fe700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7faa10071890 0x7faa100b2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:33228/0 (socket says 172.21.15.72:33228) 2023-03-24T23:47:18.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.186+0000 7faa14bd9700 1 -- 172.21.15.72:0/2828111128 learned_addr learned my addr 172.21.15.72:0/2828111128 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-03-24T23:47:18.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.186+0000 7faa0f7fe700 1 -- 172.21.15.72:0/2828111128 >> v2:172.21.15.72:3300/0 conn(0x7faa10071890 msgr2=0x7faa100b2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-03-24T23:47:18.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.186+0000 7faa0f7fe700 1 -- 172.21.15.72:0/2828111128 >> v2:172.21.15.72:3300/0 conn(0x7faa10071890 msgr2=0x7faa100b2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-03-24T23:47:18.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.186+0000 7faa0f7fe700 1 --2- 172.21.15.72:0/2828111128 >> v2:172.21.15.72:3300/0 conn(0x7faa10071890 0x7faa100b2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-03-24T23:47:18.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.186+0000 7faa0ffff700 1 --2- 172.21.15.72:0/2828111128 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa1015cd30 0x7faa100d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:18.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.186+0000 7faa0f7fe700 1 --2- 172.21.15.72:0/2828111128 >> v2:172.21.15.72:3300/0 conn(0x7faa10071890 0x7faa100b2510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-03-24T23:47:18.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.186+0000 7faa0ffff700 1 -- 172.21.15.72:0/2828111128 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa1015cd30 msgr2=0x7faa100d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-03-24T23:47:18.190 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.186+0000 7faa0ffff700 1 -- 172.21.15.72:0/2828111128 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa1015cd30 msgr2=0x7faa100d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-03-24T23:47:18.190 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.186+0000 7faa0ffff700 1 --2- 172.21.15.72:0/2828111128 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa1015cd30 0x7faa100d00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-03-24T23:47:18.190 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.186+0000 7faa0ffff700 1 --2- 172.21.15.72:0/2828111128 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa1015cd30 0x7faa100d00c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-03-24T23:47:18.368 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.367+0000 7faa0effd700 1 -- 172.21.15.72:0/2828111128 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1897609203 0 0) 0x7faa10071220 con 0x7faa100c9f20 2023-03-24T23:47:18.369 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.367+0000 7faa0effd700 1 -- 172.21.15.72:0/2828111128 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9f80035b0 con 0x7faa100c9f20 2023-03-24T23:47:18.369 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.367+0000 7faa0effd700 1 -- 172.21.15.72:0/2828111128 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1773361361 0 0) 0x7fa9f80035b0 con 0x7faa100c9f20 2023-03-24T23:47:18.369 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.367+0000 7faa0effd700 1 -- 172.21.15.72:0/2828111128 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa1015cd30 msgr2=0x7faa100d00c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-03-24T23:47:18.369 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.367+0000 7faa0effd700 1 --2- 172.21.15.72:0/2828111128 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa1015cd30 0x7faa100d00c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:18.369 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.367+0000 7faa0effd700 1 -- 172.21.15.72:0/2828111128 >> v2:172.21.15.72:3300/0 conn(0x7faa10071890 msgr2=0x7faa100b2510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-03-24T23:47:18.369 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.367+0000 7faa0effd700 1 --2- 172.21.15.72:0/2828111128 >> v2:172.21.15.72:3300/0 conn(0x7faa10071890 0x7faa100b2510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:18.370 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.367+0000 7faa0effd700 1 -- 172.21.15.72:0/2828111128 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1006b840 con 0x7faa100c9f20 2023-03-24T23:47:18.370 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.367+0000 7faa0effd700 1 -- 172.21.15.72:0/2828111128 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 293402998 0 0) 0x7faa080019f0 con 0x7faa100c9f20 2023-03-24T23:47:18.370 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.367+0000 7faa0effd700 1 -- 172.21.15.72:0/2828111128 >> v1:172.21.15.72:6789/0 conn(0x7faa100c9f20 legacy=0x7faa10160bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:18.370 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.367+0000 7faa0effd700 1 --2- 172.21.15.72:0/2828111128 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa1015cd30 0x7fa9f8003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:18.370 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.367+0000 7faa0effd700 1 --2- 172.21.15.72:0/2828111128 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9f8004080 0x7fa9f8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:18.370 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.367+0000 7faa0effd700 1 --2- 172.21.15.72:0/2828111128 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa9f8004d90 0x7fa9f8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:18.370 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.367+0000 7faa0effd700 1 -- 172.21.15.72:0/2828111128 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1006be90 con 0x7fa9f8004d90 2023-03-24T23:47:18.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.367+0000 7faa0effd700 1 -- 172.21.15.72:0/2828111128 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa10071470 con 0x7fa9f8004080 2023-03-24T23:47:18.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.367+0000 7faa0effd700 1 -- 172.21.15.72:0/2828111128 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f800bc00 con 0x7faa1015cd30 2023-03-24T23:47:18.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.367+0000 7faa14bd9700 1 --2- 172.21.15.72:0/2828111128 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa9f8004d90 0x7fa9f8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:18.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.367+0000 7faa0ffff700 1 --2- 172.21.15.72:0/2828111128 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa1015cd30 0x7fa9f8003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:18.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.367+0000 7faa0f7fe700 1 --2- 172.21.15.72:0/2828111128 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9f8004080 0x7fa9f8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:18.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.367+0000 7faa14bd9700 1 -- 172.21.15.72:0/2828111128 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa1015cd30 msgr2=0x7fa9f8003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T23:47:18.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.367+0000 7faa14bd9700 1 --2- 172.21.15.72:0/2828111128 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa1015cd30 0x7fa9f8003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:18.372 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.367+0000 7faa14bd9700 1 -- 172.21.15.72:0/2828111128 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9f8004080 msgr2=0x7fa9f8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T23:47:18.372 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.367+0000 7faa14bd9700 1 --2- 172.21.15.72:0/2828111128 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9f8004080 0x7fa9f8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:18.372 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.367+0000 7faa14bd9700 1 -- 172.21.15.72:0/2828111128 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f80035b0 con 0x7fa9f8004d90 2023-03-24T23:47:18.372 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.367+0000 7faa14bd9700 1 --2- 172.21.15.72:0/2828111128 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa9f8004d90 0x7fa9f8005200 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7faa080013d0 tx=0x7fa9f8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36581fc64dbc7dec server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:18.372 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.368+0000 7faa0effd700 1 -- 172.21.15.72:0/2828111128 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0800a250 con 0x7fa9f8004d90 2023-03-24T23:47:18.372 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.368+0000 7faa0effd700 1 -- 172.21.15.72:0/2828111128 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa080072f0 con 0x7fa9f8004d90 2023-03-24T23:47:18.372 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.368+0000 7faa0effd700 1 -- 172.21.15.72:0/2828111128 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa080076a0 con 0x7fa9f8004d90 2023-03-24T23:47:18.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.368+0000 7faa1663c700 1 -- 172.21.15.72:0/2828111128 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa9f8004d90 msgr2=0x7fa9f8005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:18.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.368+0000 7faa1663c700 1 --2- 172.21.15.72:0/2828111128 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa9f8004d90 0x7fa9f8005200 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7faa080013d0 tx=0x7fa9f8005880 comp rx=0 tx=0).stop 2023-03-24T23:47:18.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.368+0000 7faa14bd9700 1 -- 172.21.15.72:0/2828111128 reap_dead start 2023-03-24T23:47:18.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.368+0000 7faa1663c700 1 -- 172.21.15.72:0/2828111128 shutdown_connections 2023-03-24T23:47:18.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.368+0000 7faa1663c700 1 -- 172.21.15.72:0/2828111128 >> 172.21.15.72:0/2828111128 conn(0x7faa1006d5a0 msgr2=0x7faa100cf5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:18.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.369+0000 7faa1663c700 1 -- 172.21.15.72:0/2828111128 shutdown_connections 2023-03-24T23:47:18.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.369+0000 7faa1663c700 1 -- 172.21.15.72:0/2828111128 wait complete. 2023-03-24T23:47:18.375 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.370+0000 7faa1663c700 1 Processor -- start 2023-03-24T23:47:18.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.370+0000 7faa1663c700 1 -- start start 2023-03-24T23:47:18.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.371+0000 7faa1663c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faa100c57c0 0x7faa100c7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:18.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.371+0000 7faa1663c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa100c5b90 0x7faa101f2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:18.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.371+0000 7faa1663c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa1015a530 0x7faa1015a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:18.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.371+0000 7faa1663c700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa100c7920 con 0x7faa100c57c0 2023-03-24T23:47:18.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.371+0000 7faa1663c700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1015aee0 con 0x7faa100c5b90 2023-03-24T23:47:18.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.371+0000 7faa1663c700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa101f2890 con 0x7faa1015a530 2023-03-24T23:47:18.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.371+0000 7faa0ffff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faa100c57c0 0x7faa100c7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:18.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.371+0000 7faa0ffff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faa100c57c0 0x7faa100c7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:33258/0 (socket says 172.21.15.72:33258) 2023-03-24T23:47:18.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.371+0000 7faa0ffff700 1 -- 172.21.15.72:0/1164294614 learned_addr learned my addr 172.21.15.72:0/1164294614 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-03-24T23:47:18.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.371+0000 7faa14bd9700 1 --2- 172.21.15.72:0/1164294614 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa1015a530 0x7faa1015a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:18.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.371+0000 7faa0f7fe700 1 --2- 172.21.15.72:0/1164294614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa100c5b90 0x7faa101f2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:18.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.371+0000 7faa0ffff700 1 -- 172.21.15.72:0/1164294614 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa1015a530 msgr2=0x7faa1015a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:18.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.371+0000 7faa0ffff700 1 --2- 172.21.15.72:0/1164294614 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa1015a530 0x7faa1015a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:18.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.371+0000 7faa0ffff700 1 -- 172.21.15.72:0/1164294614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa100c5b90 msgr2=0x7faa101f2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:18.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.371+0000 7faa0ffff700 1 --2- 172.21.15.72:0/1164294614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa100c5b90 0x7faa101f2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:18.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.371+0000 7faa0ffff700 1 -- 172.21.15.72:0/1164294614 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1015d100 con 0x7faa100c57c0 2023-03-24T23:47:18.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.371+0000 7faa0ffff700 1 --2- 172.21.15.72:0/1164294614 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faa100c57c0 0x7faa100c7060 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faa00010a20 tx=0x7faa00010d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:18.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.371+0000 7faa0cff9700 1 -- 172.21.15.72:0/1164294614 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa00006ea0 con 0x7faa100c57c0 2023-03-24T23:47:18.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.371+0000 7faa0cff9700 1 -- 172.21.15.72:0/1164294614 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0000ea70 con 0x7faa100c57c0 2023-03-24T23:47:18.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.371+0000 7faa1663c700 1 -- 172.21.15.72:0/1164294614 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa1006bb20 con 0x7faa100c57c0 2023-03-24T23:47:18.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.371+0000 7faa0cff9700 1 -- 172.21.15.72:0/1164294614 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa00009730 con 0x7faa100c57c0 2023-03-24T23:47:18.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.371+0000 7faa1663c700 1 -- 172.21.15.72:0/1164294614 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa100b2a50 con 0x7faa100c57c0 2023-03-24T23:47:18.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.372+0000 7faa0cff9700 1 -- 172.21.15.72:0/1164294614 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7faa00015470 con 0x7faa100c57c0 2023-03-24T23:47:18.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.372+0000 7faa0cff9700 1 -- 172.21.15.72:0/1164294614 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7faa00016420 con 0x7faa100c57c0 2023-03-24T23:47:18.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.372+0000 7faa1663c700 1 -- 172.21.15.72:0/1164294614 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa1004e850 con 0x7faa100c57c0 2023-03-24T23:47:18.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.374+0000 7faa0cff9700 1 -- 172.21.15.72:0/1164294614 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7faa0003eb10 con 0x7faa100c57c0 2023-03-24T23:47:18.447 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.447+0000 7faa1663c700 1 -- 172.21.15.72:0/1164294614 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7faa100667b0 con 0x7faa100c57c0 2023-03-24T23:47:18.449 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.449+0000 7faa0cff9700 1 -- 172.21.15.72:0/1164294614 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7faa0002d020 con 0x7faa100c57c0 2023-03-24T23:47:18.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.449+0000 7faa1663c700 1 -- 172.21.15.72:0/1164294614 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faa100c57c0 msgr2=0x7faa100c7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:18.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.449+0000 7faa1663c700 1 --2- 172.21.15.72:0/1164294614 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faa100c57c0 0x7faa100c7060 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faa00010a20 tx=0x7faa00010d30 comp rx=0 tx=0).stop 2023-03-24T23:47:18.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.449+0000 7faa1663c700 1 -- 172.21.15.72:0/1164294614 shutdown_connections 2023-03-24T23:47:18.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.449+0000 7faa1663c700 1 --2- 172.21.15.72:0/1164294614 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa1015a530 0x7faa1015a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:18.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.449+0000 7faa1663c700 1 --2- 172.21.15.72:0/1164294614 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faa100c57c0 0x7faa100c7060 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:18.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.449+0000 7faa1663c700 1 --2- 172.21.15.72:0/1164294614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa100c5b90 0x7faa101f2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:18.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.449+0000 7faa1663c700 1 -- 172.21.15.72:0/1164294614 >> 172.21.15.72:0/1164294614 conn(0x7faa1006d5a0 msgr2=0x7faa10159c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:18.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.450+0000 7faa1663c700 1 -- 172.21.15.72:0/1164294614 shutdown_connections 2023-03-24T23:47:18.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:18.450+0000 7faa1663c700 1 -- 172.21.15.72:0/1164294614 wait complete. 2023-03-24T23:47:18.452 INFO:teuthology.orchestra.run.smithi072.stderr:adjusted tunables profile to default 2023-03-24T23:47:18.461 INFO:tasks.ceph:check_enable_crimson: False 2023-03-24T23:47:18.462 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-03-24T23:47:18.462 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-24T23:47:18.462 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-03-24T23:47:18.489 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-24T23:47:18.489 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-03-24T23:47:18.556 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-24T23:47:18.556 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-03-24T23:47:18.623 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-24T23:47:18.623 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-03-24T23:47:18.689 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-03-24T23:47:18.690 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-03-24T23:47:18.717 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-03-24T23:47:18.718 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-03-24T23:47:18.786 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-03-24T23:47:18.787 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-03-24T23:47:18.853 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-03-24T23:47:18.854 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-03-24T23:47:18.921 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new 993abb78-a510-4d76-af32-9015a2b3e5d2 0 2023-03-24T23:47:19.085 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.084+0000 7f92c8823700 1 Processor -- start 2023-03-24T23:47:19.085 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.084+0000 7f92c8823700 1 -- start start 2023-03-24T23:47:19.087 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.085+0000 7f92c8823700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f92c406e090 0x7f92c406e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:19.087 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.085+0000 7f92c8823700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f92c40cfbc0 0x7f92c406b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:19.087 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.085+0000 7f92c8823700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92c406e9a0 con 0x7f92c406cf50 2023-03-24T23:47:19.087 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.085+0000 7f92c8823700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92c406cb30 con 0x7f92c406e090 2023-03-24T23:47:19.087 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.085+0000 7f92c8823700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92c406beb0 con 0x7f92c40cfbc0 2023-03-24T23:47:19.087 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.085+0000 7f92c1d9b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f92c40cfbc0 0x7f92c406b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:19.088 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.085+0000 7f92c1d9b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f92c40cfbc0 0x7f92c406b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50720/0 (socket says 172.21.15.146:50720) 2023-03-24T23:47:19.088 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.085+0000 7f92c2d9d700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f92c406cf50 0x7f92c40adaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:49648/0 (socket says 172.21.15.146:49648) 2023-03-24T23:47:19.088 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.085+0000 7f92c1d9b700 1 -- 172.21.15.146:0/681627407 learned_addr learned my addr 172.21.15.146:0/681627407 (peer_addr_for_me v2:172.21.15.146:0/0) 2023-03-24T23:47:19.088 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.085+0000 7f92c259c700 1 --2- 172.21.15.146:0/681627407 >> v2:172.21.15.72:3300/0 conn(0x7f92c406e090 0x7f92c406e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:19.088 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.085+0000 7f92c1d9b700 1 -- 172.21.15.146:0/681627407 >> v2:172.21.15.72:3300/0 conn(0x7f92c406e090 msgr2=0x7f92c406e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T23:47:19.088 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.085+0000 7f92c1d9b700 1 --2- 172.21.15.146:0/681627407 >> v2:172.21.15.72:3300/0 conn(0x7f92c406e090 0x7f92c406e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:19.088 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.085+0000 7f92c1d9b700 1 -- 172.21.15.146:0/681627407 >> v1:172.21.15.146:6789/0 conn(0x7f92c406cf50 legacy=0x7f92c40adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:19.088 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.085+0000 7f92c159a700 1 -- 172.21.15.146:0/681627407 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1240865601 0 0) 0x7f92c406e9a0 con 0x7f92c406cf50 2023-03-24T23:47:19.089 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.085+0000 7f92c1d9b700 1 -- 172.21.15.146:0/681627407 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c406c190 con 0x7f92c40cfbc0 2023-03-24T23:47:19.089 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.085+0000 7f92c1d9b700 1 --2- 172.21.15.146:0/681627407 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92c40cfbc0 0x7f92c406b5b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f92c40b3a90 tx=0x7f92b8009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b78a37e948936f74 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:19.089 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.086+0000 7f92c159a700 1 -- 172.21.15.146:0/681627407 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b8011830 con 0x7f92c40cfbc0 2023-03-24T23:47:19.089 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.086+0000 7f92c159a700 1 -- 172.21.15.146:0/681627407 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b8011990 con 0x7f92c40cfbc0 2023-03-24T23:47:19.089 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.086+0000 7f92c159a700 1 -- 172.21.15.146:0/681627407 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b8011d20 con 0x7f92c40cfbc0 2023-03-24T23:47:19.091 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.086+0000 7f92c8823700 1 -- 172.21.15.146:0/681627407 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92c40cfbc0 msgr2=0x7f92c406b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:19.091 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.086+0000 7f92c8823700 1 --2- 172.21.15.146:0/681627407 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92c40cfbc0 0x7f92c406b5b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f92c40b3a90 tx=0x7f92b8009bd0 comp rx=0 tx=0).stop 2023-03-24T23:47:19.092 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.086+0000 7f92c8823700 1 -- 172.21.15.146:0/681627407 shutdown_connections 2023-03-24T23:47:19.092 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.086+0000 7f92c8823700 1 --2- 172.21.15.146:0/681627407 >> v2:172.21.15.72:3300/0 conn(0x7f92c406e090 0x7f92c406e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:19.092 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.086+0000 7f92c8823700 1 --2- 172.21.15.146:0/681627407 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92c40cfbc0 0x7f92c406b5b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:19.092 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.086+0000 7f92c8823700 1 -- 172.21.15.146:0/681627407 >> 172.21.15.146:0/681627407 conn(0x7f92c409a290 msgr2=0x7f92c40d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:19.092 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.087+0000 7f92c8823700 1 -- 172.21.15.146:0/681627407 shutdown_connections 2023-03-24T23:47:19.092 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.087+0000 7f92c8823700 1 -- 172.21.15.146:0/681627407 wait complete. 2023-03-24T23:47:19.092 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.088+0000 7f92c8823700 1 Processor -- start 2023-03-24T23:47:19.092 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.088+0000 7f92c8823700 1 -- start start 2023-03-24T23:47:19.093 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.089+0000 7f92c8823700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92c406cf50 0x7f92c40afd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:19.093 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.089+0000 7f92c8823700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92c406e090 0x7f92c4158350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:19.093 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.089+0000 7f92c8823700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f92c40cfbc0 0x7f92c415e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:19.093 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.089+0000 7f92c8823700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c40b57f0 con 0x7f92c40cfbc0 2023-03-24T23:47:19.093 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.089+0000 7f92c8823700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c40b5ad0 con 0x7f92c406cf50 2023-03-24T23:47:19.093 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.089+0000 7f92c8823700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92c40b5db0 con 0x7f92c406e090 2023-03-24T23:47:19.093 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.089+0000 7f92c1d9b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92c406e090 0x7f92c4158350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:19.094 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.089+0000 7f92c1d9b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92c406e090 0x7f92c4158350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.146:38034/0 (socket says 172.21.15.146:38034) 2023-03-24T23:47:19.094 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.089+0000 7f92c1d9b700 1 -- 172.21.15.146:0/1514348436 learned_addr learned my addr 172.21.15.146:0/1514348436 (peer_addr_for_me v2:172.21.15.146:0/0) 2023-03-24T23:47:19.094 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.089+0000 7f92c259c700 1 --2- 172.21.15.146:0/1514348436 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92c406cf50 0x7f92c40afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:19.094 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.089+0000 7f92c2d9d700 1 --2- 172.21.15.146:0/1514348436 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f92c40cfbc0 0x7f92c415e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:19.094 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.089+0000 7f92c1d9b700 1 -- 172.21.15.146:0/1514348436 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92c406cf50 msgr2=0x7f92c40afd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:19.094 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.089+0000 7f92c1d9b700 1 --2- 172.21.15.146:0/1514348436 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92c406cf50 0x7f92c40afd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:19.094 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.089+0000 7f92c1d9b700 1 -- 172.21.15.146:0/1514348436 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f92c40cfbc0 msgr2=0x7f92c415e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:19.094 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.089+0000 7f92c1d9b700 1 --2- 172.21.15.146:0/1514348436 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f92c40cfbc0 0x7f92c415e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:19.095 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.089+0000 7f92c1d9b700 1 -- 172.21.15.146:0/1514348436 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b8009550 con 0x7f92c406e090 2023-03-24T23:47:19.095 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.089+0000 7f92c1d9b700 1 --2- 172.21.15.146:0/1514348436 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92c406e090 0x7f92c4158350 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f92c406bff0 tx=0x7f92b8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:19.095 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.090+0000 7f92b2ffd700 1 -- 172.21.15.146:0/1514348436 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b8004510 con 0x7f92c406e090 2023-03-24T23:47:19.095 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.090+0000 7f92b2ffd700 1 -- 172.21.15.146:0/1514348436 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b80036a0 con 0x7f92c406e090 2023-03-24T23:47:19.095 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.090+0000 7f92c8823700 1 -- 172.21.15.146:0/1514348436 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92c406c750 con 0x7f92c406e090 2023-03-24T23:47:19.095 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.090+0000 7f92b2ffd700 1 -- 172.21.15.146:0/1514348436 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b8003a30 con 0x7f92c406e090 2023-03-24T23:47:19.095 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.090+0000 7f92c8823700 1 -- 172.21.15.146:0/1514348436 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92c406c470 con 0x7f92c406e090 2023-03-24T23:47:19.095 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.090+0000 7f92b2ffd700 1 -- 172.21.15.146:0/1514348436 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f92b801a430 con 0x7f92c406e090 2023-03-24T23:47:19.096 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.090+0000 7f92b2ffd700 1 -- 172.21.15.146:0/1514348436 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f92b8021030 con 0x7f92c406e090 2023-03-24T23:47:19.096 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.091+0000 7f92c8823700 1 -- 172.21.15.146:0/1514348436 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92c406baf0 con 0x7f92c406e090 2023-03-24T23:47:19.096 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.093+0000 7f92b2ffd700 1 -- 172.21.15.146:0/1514348436 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f92b802b060 con 0x7f92c406e090 2023-03-24T23:47:19.191 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.190+0000 7f92c8823700 1 -- 172.21.15.146:0/1514348436 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "osd new", "uuid": "993abb78-a510-4d76-af32-9015a2b3e5d2", "id": 0} v 0) v1 -- 0x7f92c404e850 con 0x7f92c406e090 2023-03-24T23:47:19.197 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.196+0000 7f92b2ffd700 1 -- 172.21.15.146:0/1514348436 <== mon.2 v2:172.21.15.146:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "993abb78-a510-4d76-af32-9015a2b3e5d2", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f92b8015020 con 0x7f92c406e090 2023-03-24T23:47:19.197 INFO:teuthology.orchestra.run.smithi146.stdout:0 2023-03-24T23:47:19.198 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.197+0000 7f92c8823700 1 -- 172.21.15.146:0/1514348436 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92c406e090 msgr2=0x7f92c4158350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:19.198 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.197+0000 7f92c8823700 1 --2- 172.21.15.146:0/1514348436 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92c406e090 0x7f92c4158350 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f92c406bff0 tx=0x7f92b8004750 comp rx=0 tx=0).stop 2023-03-24T23:47:19.199 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.197+0000 7f92c8823700 1 -- 172.21.15.146:0/1514348436 shutdown_connections 2023-03-24T23:47:19.199 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.197+0000 7f92c8823700 1 --2- 172.21.15.146:0/1514348436 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f92c40cfbc0 0x7f92c415e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:19.199 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.197+0000 7f92c8823700 1 --2- 172.21.15.146:0/1514348436 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92c406cf50 0x7f92c40afd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:19.199 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.197+0000 7f92c8823700 1 --2- 172.21.15.146:0/1514348436 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92c406e090 0x7f92c4158350 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:19.199 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.197+0000 7f92c8823700 1 -- 172.21.15.146:0/1514348436 >> 172.21.15.146:0/1514348436 conn(0x7f92c409a290 msgr2=0x7f92c40adcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:19.200 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.197+0000 7f92c8823700 1 -- 172.21.15.146:0/1514348436 shutdown_connections 2023-03-24T23:47:19.200 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.197+0000 7f92c8823700 1 -- 172.21.15.146:0/1514348436 wait complete. 2023-03-24T23:47:19.209 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new ddfa95f0-f08c-4dce-b961-2e3a9f4bcb3d 1 2023-03-24T23:47:19.331 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.330+0000 7f98d15a4700 1 Processor -- start 2023-03-24T23:47:19.332 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.330+0000 7f98d15a4700 1 -- start start 2023-03-24T23:47:19.333 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.331+0000 7f98d15a4700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f98cc0d01e0 0x7f98cc0d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:19.333 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.331+0000 7f98d15a4700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98cc0d2830 con 0x7f98cc0b23c0 2023-03-24T23:47:19.333 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.331+0000 7f98d15a4700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98cc0d2df0 con 0x7f98cc0b5f80 2023-03-24T23:47:19.333 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.331+0000 7f98d15a4700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98cc0b2790 con 0x7f98cc0d01e0 2023-03-24T23:47:19.333 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.331+0000 7f98ca7fc700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f98cc0b5f80 0x7f98cc0cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:49660/0 (socket says 172.21.15.146:49660) 2023-03-24T23:47:19.333 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.331+0000 7f98ca7fc700 1 -- 172.21.15.146:0/3629591353 learned_addr learned my addr 172.21.15.146:0/3629591353 (peer_addr_for_me v1:172.21.15.146:0/0) 2023-03-24T23:47:19.334 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.331+0000 7f98cb7fe700 1 --2- 172.21.15.146:0/3629591353 >> v2:172.21.15.146:3300/0 conn(0x7f98cc0d01e0 0x7f98cc0d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:19.334 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.331+0000 7f98c9ffb700 1 -- 172.21.15.146:0/3629591353 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4173375591 0 0) 0x7f98cc0d2830 con 0x7f98cc0b23c0 2023-03-24T23:47:19.334 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.331+0000 7f98cb7fe700 1 -- 172.21.15.146:0/3629591353 >> v1:172.21.15.146:6789/0 conn(0x7f98cc0b5f80 legacy=0x7f98cc0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:19.334 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.331+0000 7f98cb7fe700 1 -- 172.21.15.146:0/3629591353 >> v1:172.21.15.72:6789/0 conn(0x7f98cc0b23c0 legacy=0x7f98cc0d1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:19.334 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.331+0000 7f98cb7fe700 1 -- 172.21.15.146:0/3629591353 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98cc0b6400 con 0x7f98cc0d01e0 2023-03-24T23:47:19.334 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.332+0000 7f98cb7fe700 1 --2- 172.21.15.146:0/3629591353 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98cc0d01e0 0x7f98cc0d3280 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f98c4009840 tx=0x7f98c4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e261388fb00f8253 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:19.334 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.332+0000 7f98c9ffb700 1 -- 172.21.15.146:0/3629591353 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c4011830 con 0x7f98cc0d01e0 2023-03-24T23:47:19.334 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.332+0000 7f98c9ffb700 1 -- 172.21.15.146:0/3629591353 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c4011990 con 0x7f98cc0d01e0 2023-03-24T23:47:19.335 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.332+0000 7f98c9ffb700 1 -- 172.21.15.146:0/3629591353 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c4011d20 con 0x7f98cc0d01e0 2023-03-24T23:47:19.337 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.332+0000 7f98d15a4700 1 -- 172.21.15.146:0/3629591353 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98cc0d01e0 msgr2=0x7f98cc0d3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:19.337 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.332+0000 7f98d15a4700 1 --2- 172.21.15.146:0/3629591353 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98cc0d01e0 0x7f98cc0d3280 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f98c4009840 tx=0x7f98c4009b50 comp rx=0 tx=0).stop 2023-03-24T23:47:19.338 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.332+0000 7f98d15a4700 1 -- 172.21.15.146:0/3629591353 shutdown_connections 2023-03-24T23:47:19.338 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.332+0000 7f98d15a4700 1 --2- 172.21.15.146:0/3629591353 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98cc0d01e0 0x7f98cc0d3280 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:19.338 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.332+0000 7f98d15a4700 1 -- 172.21.15.146:0/3629591353 >> 172.21.15.146:0/3629591353 conn(0x7f98cc066140 msgr2=0x7f98cc066540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:19.338 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.332+0000 7f98d15a4700 1 -- 172.21.15.146:0/3629591353 shutdown_connections 2023-03-24T23:47:19.338 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.332+0000 7f98d15a4700 1 -- 172.21.15.146:0/3629591353 wait complete. 2023-03-24T23:47:19.338 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.334+0000 7f98d15a4700 1 Processor -- start 2023-03-24T23:47:19.338 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.334+0000 7f98d15a4700 1 -- start start 2023-03-24T23:47:19.338 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.334+0000 7f98d15a4700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98cc0b23c0 0x7f98cc15a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:19.339 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.334+0000 7f98d15a4700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98cc0b5f80 0x7f98cc0cbb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:19.339 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.334+0000 7f98d15a4700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98cc0d01e0 0x7f98cc1f57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:19.339 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.334+0000 7f98d15a4700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc0cc510 con 0x7f98cc0d01e0 2023-03-24T23:47:19.339 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.334+0000 7f98d15a4700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc0c6ba0 con 0x7f98cc0b23c0 2023-03-24T23:47:19.339 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.334+0000 7f98d15a4700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc0c6e80 con 0x7f98cc0b5f80 2023-03-24T23:47:19.339 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.335+0000 7f98caffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98cc0b23c0 0x7f98cc15a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:19.339 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.335+0000 7f98ca7fc700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98cc0b5f80 0x7f98cc0cbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:19.340 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.335+0000 7f98caffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98cc0b23c0 0x7f98cc15a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50754/0 (socket says 172.21.15.146:50754) 2023-03-24T23:47:19.340 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.335+0000 7f98caffd700 1 -- 172.21.15.146:0/2547430249 learned_addr learned my addr 172.21.15.146:0/2547430249 (peer_addr_for_me v2:172.21.15.146:0/0) 2023-03-24T23:47:19.340 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.335+0000 7f98cb7fe700 1 --2- 172.21.15.146:0/2547430249 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98cc0d01e0 0x7f98cc1f57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:19.340 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.335+0000 7f98ca7fc700 1 -- 172.21.15.146:0/2547430249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98cc0b23c0 msgr2=0x7f98cc15a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:19.340 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.335+0000 7f98ca7fc700 1 --2- 172.21.15.146:0/2547430249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98cc0b23c0 0x7f98cc15a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:19.340 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.335+0000 7f98ca7fc700 1 -- 172.21.15.146:0/2547430249 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98cc0d01e0 msgr2=0x7f98cc1f57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:19.340 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.335+0000 7f98ca7fc700 1 --2- 172.21.15.146:0/2547430249 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98cc0d01e0 0x7f98cc1f57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:19.340 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.335+0000 7f98ca7fc700 1 -- 172.21.15.146:0/2547430249 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c40094a0 con 0x7f98cc0b5f80 2023-03-24T23:47:19.341 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.335+0000 7f98caffd700 1 --2- 172.21.15.146:0/2547430249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98cc0b23c0 0x7f98cc15a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-24T23:47:19.341 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.335+0000 7f98cb7fe700 1 --2- 172.21.15.146:0/2547430249 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98cc0d01e0 0x7f98cc1f57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-24T23:47:19.341 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.335+0000 7f98ca7fc700 1 --2- 172.21.15.146:0/2547430249 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98cc0b5f80 0x7f98cc0cbb50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f98c000c500 tx=0x7f98c000c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:19.341 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.335+0000 7f98b37fe700 1 -- 172.21.15.146:0/2547430249 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c0005d70 con 0x7f98cc0b5f80 2023-03-24T23:47:19.341 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.335+0000 7f98b37fe700 1 -- 172.21.15.146:0/2547430249 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c00070d0 con 0x7f98cc0b5f80 2023-03-24T23:47:19.341 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.335+0000 7f98d15a4700 1 -- 172.21.15.146:0/2547430249 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98cc0cdc00 con 0x7f98cc0b5f80 2023-03-24T23:47:19.341 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.335+0000 7f98b37fe700 1 -- 172.21.15.146:0/2547430249 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c0007440 con 0x7f98cc0b5f80 2023-03-24T23:47:19.341 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.335+0000 7f98d15a4700 1 -- 172.21.15.146:0/2547430249 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98cc0cd920 con 0x7f98cc0b5f80 2023-03-24T23:47:19.342 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.336+0000 7f98b37fe700 1 -- 172.21.15.146:0/2547430249 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f98c001c440 con 0x7f98cc0b5f80 2023-03-24T23:47:19.342 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.336+0000 7f98d15a4700 1 -- 172.21.15.146:0/2547430249 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98cc0d2a80 con 0x7f98cc0b5f80 2023-03-24T23:47:19.342 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.336+0000 7f98b37fe700 1 -- 172.21.15.146:0/2547430249 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f98c00127b0 con 0x7f98cc0b5f80 2023-03-24T23:47:19.342 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.340+0000 7f98b37fe700 1 -- 172.21.15.146:0/2547430249 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f98c0021060 con 0x7f98cc0b5f80 2023-03-24T23:47:19.426 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.424+0000 7f98d15a4700 1 -- 172.21.15.146:0/2547430249 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ddfa95f0-f08c-4dce-b961-2e3a9f4bcb3d", "id": 1} v 0) v1 -- 0x7f98cc04e850 con 0x7f98cc0b5f80 2023-03-24T23:47:19.433 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.432+0000 7f98b37fe700 1 -- 172.21.15.146:0/2547430249 <== mon.2 v2:172.21.15.146:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ddfa95f0-f08c-4dce-b961-2e3a9f4bcb3d", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f98c0017070 con 0x7f98cc0b5f80 2023-03-24T23:47:19.434 INFO:teuthology.orchestra.run.smithi146.stdout:1 2023-03-24T23:47:19.435 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.433+0000 7f98d15a4700 1 -- 172.21.15.146:0/2547430249 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98cc0b5f80 msgr2=0x7f98cc0cbb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:19.435 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.433+0000 7f98d15a4700 1 --2- 172.21.15.146:0/2547430249 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98cc0b5f80 0x7f98cc0cbb50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f98c000c500 tx=0x7f98c000c810 comp rx=0 tx=0).stop 2023-03-24T23:47:19.435 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.433+0000 7f98d15a4700 1 -- 172.21.15.146:0/2547430249 shutdown_connections 2023-03-24T23:47:19.435 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.433+0000 7f98d15a4700 1 --2- 172.21.15.146:0/2547430249 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98cc0d01e0 0x7f98cc1f57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:19.436 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.433+0000 7f98d15a4700 1 --2- 172.21.15.146:0/2547430249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98cc0b23c0 0x7f98cc15a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:19.436 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.433+0000 7f98d15a4700 1 --2- 172.21.15.146:0/2547430249 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98cc0b5f80 0x7f98cc0cbb50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:19.436 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.433+0000 7f98d15a4700 1 -- 172.21.15.146:0/2547430249 >> 172.21.15.146:0/2547430249 conn(0x7f98cc066140 msgr2=0x7f98cc0d1260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:19.436 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.433+0000 7f98d15a4700 1 -- 172.21.15.146:0/2547430249 shutdown_connections 2023-03-24T23:47:19.436 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.433+0000 7f98d15a4700 1 -- 172.21.15.146:0/2547430249 wait complete. 2023-03-24T23:47:19.446 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new cfcf995d-5a1f-47c1-9691-858f305b1f47 2 2023-03-24T23:47:19.569 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.568+0000 7f3e9d6c9700 1 Processor -- start 2023-03-24T23:47:19.570 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.568+0000 7f3e9d6c9700 1 -- start start 2023-03-24T23:47:19.571 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.569+0000 7f3e9d6c9700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3e981614a0 0x7f3e98070f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:19.572 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.569+0000 7f3e9d6c9700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f3e9806c2d0 0x7f3e980714b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:19.572 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.569+0000 7f3e9d6c9700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e981607c0 con 0x7f3e981610d0 2023-03-24T23:47:19.572 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.569+0000 7f3e9d6c9700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e9815d0a0 con 0x7f3e9806c2d0 2023-03-24T23:47:19.572 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.569+0000 7f3e9d6c9700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e980af500 con 0x7f3e981614a0 2023-03-24T23:47:19.572 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.569+0000 7f3e967fc700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3e981614a0 0x7f3e98070f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:19.572 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.569+0000 7f3e967fc700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3e981614a0 0x7f3e98070f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50770/0 (socket says 172.21.15.146:50770) 2023-03-24T23:47:19.572 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.569+0000 7f3e967fc700 1 -- 172.21.15.146:0/1042357170 learned_addr learned my addr 172.21.15.146:0/1042357170 (peer_addr_for_me v2:172.21.15.146:0/0) 2023-03-24T23:47:19.573 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.569+0000 7f3e977fe700 1 --2- 172.21.15.146:0/1042357170 >> v2:172.21.15.72:3300/0 conn(0x7f3e9806c2d0 0x7f3e980714b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:19.573 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.569+0000 7f3e967fc700 1 -- 172.21.15.146:0/1042357170 >> v2:172.21.15.72:3300/0 conn(0x7f3e9806c2d0 msgr2=0x7f3e980714b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T23:47:19.573 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.569+0000 7f3e967fc700 1 --2- 172.21.15.146:0/1042357170 >> v2:172.21.15.72:3300/0 conn(0x7f3e9806c2d0 0x7f3e980714b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:19.573 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.569+0000 7f3e967fc700 1 -- 172.21.15.146:0/1042357170 >> v1:172.21.15.146:6789/0 conn(0x7f3e981610d0 legacy=0x7f3e980aeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:19.573 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.570+0000 7f3e967fc700 1 -- 172.21.15.146:0/1042357170 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e9806e010 con 0x7f3e981614a0 2023-03-24T23:47:19.573 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.570+0000 7f3e95ffb700 1 -- 172.21.15.146:0/1042357170 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2367455838 0 0) 0x7f3e981607c0 con 0x7f3e981610d0 2023-03-24T23:47:19.573 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.570+0000 7f3e977fe700 1 --2- 172.21.15.146:0/1042357170 >> v2:172.21.15.72:3300/0 conn(0x7f3e9806c2d0 0x7f3e980714b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-24T23:47:19.573 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.570+0000 7f3e967fc700 1 --2- 172.21.15.146:0/1042357170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e981614a0 0x7f3e98070f70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3e98072250 tx=0x7f3e8c009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a35216d8376dd727 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:19.574 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.570+0000 7f3e95ffb700 1 -- 172.21.15.146:0/1042357170 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c00b870 con 0x7f3e981614a0 2023-03-24T23:47:19.574 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.570+0000 7f3e95ffb700 1 -- 172.21.15.146:0/1042357170 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e8c00b9d0 con 0x7f3e981614a0 2023-03-24T23:47:19.574 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.570+0000 7f3e95ffb700 1 -- 172.21.15.146:0/1042357170 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c00bd60 con 0x7f3e981614a0 2023-03-24T23:47:19.574 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.570+0000 7f3e9d6c9700 1 -- 172.21.15.146:0/1042357170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e981614a0 msgr2=0x7f3e98070f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:19.574 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.570+0000 7f3e9d6c9700 1 --2- 172.21.15.146:0/1042357170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e981614a0 0x7f3e98070f70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3e98072250 tx=0x7f3e8c009bd0 comp rx=0 tx=0).stop 2023-03-24T23:47:19.574 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.570+0000 7f3e9d6c9700 1 -- 172.21.15.146:0/1042357170 shutdown_connections 2023-03-24T23:47:19.574 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.570+0000 7f3e9d6c9700 1 --2- 172.21.15.146:0/1042357170 >> v2:172.21.15.72:3300/0 conn(0x7f3e9806c2d0 0x7f3e980714b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:19.575 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.570+0000 7f3e9d6c9700 1 --2- 172.21.15.146:0/1042357170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e981614a0 0x7f3e98070f70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:19.575 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.570+0000 7f3e9d6c9700 1 -- 172.21.15.146:0/1042357170 >> 172.21.15.146:0/1042357170 conn(0x7f3e980d0a00 msgr2=0x7f3e980cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:19.575 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.570+0000 7f3e9d6c9700 1 -- 172.21.15.146:0/1042357170 shutdown_connections 2023-03-24T23:47:19.575 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.571+0000 7f3e9d6c9700 1 -- 172.21.15.146:0/1042357170 wait complete. 2023-03-24T23:47:19.577 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.572+0000 7f3e9d6c9700 1 Processor -- start 2023-03-24T23:47:19.577 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.572+0000 7f3e9d6c9700 1 -- start start 2023-03-24T23:47:19.577 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.573+0000 7f3e9d6c9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3e9806c2d0 0x7f3e9806ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:19.577 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.573+0000 7f3e9d6c9700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e981610d0 0x7f3e980b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:19.578 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.573+0000 7f3e9d6c9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e981614a0 0x7f3e980c5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:19.578 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.573+0000 7f3e9d6c9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e980b4a60 con 0x7f3e9806c2d0 2023-03-24T23:47:19.578 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.573+0000 7f3e9d6c9700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e980b4d40 con 0x7f3e981614a0 2023-03-24T23:47:19.578 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.573+0000 7f3e9d6c9700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e980b5020 con 0x7f3e981610d0 2023-03-24T23:47:19.578 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.573+0000 7f3e977fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e981614a0 0x7f3e980c5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:19.578 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.573+0000 7f3e967fc700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e981610d0 0x7f3e980b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:19.578 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.573+0000 7f3e977fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e981614a0 0x7f3e980c5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50780/0 (socket says 172.21.15.146:50780) 2023-03-24T23:47:19.578 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.573+0000 7f3e96ffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3e9806c2d0 0x7f3e9806ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:19.579 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.573+0000 7f3e977fe700 1 -- 172.21.15.146:0/1030158922 learned_addr learned my addr 172.21.15.146:0/1030158922 (peer_addr_for_me v2:172.21.15.146:0/0) 2023-03-24T23:47:19.579 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.573+0000 7f3e977fe700 1 -- 172.21.15.146:0/1030158922 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e981610d0 msgr2=0x7f3e980b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:19.579 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.573+0000 7f3e977fe700 1 --2- 172.21.15.146:0/1030158922 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e981610d0 0x7f3e980b40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:19.579 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.573+0000 7f3e977fe700 1 -- 172.21.15.146:0/1030158922 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3e9806c2d0 msgr2=0x7f3e9806ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:19.579 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.573+0000 7f3e977fe700 1 --2- 172.21.15.146:0/1030158922 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3e9806c2d0 0x7f3e9806ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:19.579 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.573+0000 7f3e977fe700 1 -- 172.21.15.146:0/1030158922 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e8c009550 con 0x7f3e981614a0 2023-03-24T23:47:19.579 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.573+0000 7f3e967fc700 1 --2- 172.21.15.146:0/1030158922 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e981610d0 0x7f3e980b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-24T23:47:19.580 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.573+0000 7f3e977fe700 1 --2- 172.21.15.146:0/1030158922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e981614a0 0x7f3e980c5c40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3e90009f70 tx=0x7f3e9000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:19.580 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.573+0000 7f3e7f7fe700 1 -- 172.21.15.146:0/1030158922 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9000dd80 con 0x7f3e981614a0 2023-03-24T23:47:19.580 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.573+0000 7f3e7f7fe700 1 -- 172.21.15.146:0/1030158922 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9000f040 con 0x7f3e981614a0 2023-03-24T23:47:19.580 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.573+0000 7f3e9d6c9700 1 -- 172.21.15.146:0/1030158922 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e9806e5d0 con 0x7f3e981614a0 2023-03-24T23:47:19.580 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.573+0000 7f3e7f7fe700 1 -- 172.21.15.146:0/1030158922 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e90003c60 con 0x7f3e981614a0 2023-03-24T23:47:19.580 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.573+0000 7f3e9d6c9700 1 -- 172.21.15.146:0/1030158922 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e9806e2f0 con 0x7f3e981614a0 2023-03-24T23:47:19.580 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.574+0000 7f3e9d6c9700 1 -- 172.21.15.146:0/1030158922 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e9804e850 con 0x7f3e981614a0 2023-03-24T23:47:19.580 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.575+0000 7f3e7f7fe700 1 -- 172.21.15.146:0/1030158922 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3e90004330 con 0x7f3e981614a0 2023-03-24T23:47:19.581 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.575+0000 7f3e7f7fe700 1 -- 172.21.15.146:0/1030158922 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f3e90015860 con 0x7f3e981614a0 2023-03-24T23:47:19.581 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.577+0000 7f3e7f7fe700 1 -- 172.21.15.146:0/1030158922 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f3e9001a070 con 0x7f3e981614a0 2023-03-24T23:47:19.663 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.662+0000 7f3e9d6c9700 1 -- 172.21.15.146:0/1030158922 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd new", "uuid": "cfcf995d-5a1f-47c1-9691-858f305b1f47", "id": 2} v 0) v1 -- 0x7f3e980af140 con 0x7f3e981614a0 2023-03-24T23:47:19.671 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.670+0000 7f3e7f7fe700 1 -- 172.21.15.146:0/1030158922 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cfcf995d-5a1f-47c1-9691-858f305b1f47", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3e90003dc0 con 0x7f3e981614a0 2023-03-24T23:47:19.671 INFO:teuthology.orchestra.run.smithi146.stdout:2 2023-03-24T23:47:19.672 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.671+0000 7f3e9d6c9700 1 -- 172.21.15.146:0/1030158922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e981614a0 msgr2=0x7f3e980c5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:19.672 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.671+0000 7f3e9d6c9700 1 --2- 172.21.15.146:0/1030158922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e981614a0 0x7f3e980c5c40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3e90009f70 tx=0x7f3e9000d3b0 comp rx=0 tx=0).stop 2023-03-24T23:47:19.672 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.671+0000 7f3e9d6c9700 1 -- 172.21.15.146:0/1030158922 shutdown_connections 2023-03-24T23:47:19.673 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.671+0000 7f3e9d6c9700 1 --2- 172.21.15.146:0/1030158922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e981614a0 0x7f3e980c5c40 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:19.673 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.671+0000 7f3e9d6c9700 1 --2- 172.21.15.146:0/1030158922 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3e9806c2d0 0x7f3e9806ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:19.673 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.671+0000 7f3e9d6c9700 1 --2- 172.21.15.146:0/1030158922 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e981610d0 0x7f3e980b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:19.673 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.671+0000 7f3e9d6c9700 1 -- 172.21.15.146:0/1030158922 >> 172.21.15.146:0/1030158922 conn(0x7f3e980d0a00 msgr2=0x7f3e9815a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:19.673 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.671+0000 7f3e9d6c9700 1 -- 172.21.15.146:0/1030158922 shutdown_connections 2023-03-24T23:47:19.673 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.671+0000 7f3e9d6c9700 1 -- 172.21.15.146:0/1030158922 wait complete. 2023-03-24T23:47:19.684 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new 17c7d98c-55ba-4001-8eac-475f82cc31e1 3 2023-03-24T23:47:19.713 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-24T23:47:19.713+0000 7f18a44d5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-03-24T23:47:19.725 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-03-24T23:47:19.724+0000 7ffbd8287200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-03-24T23:47:19.783 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-24T23:47:19.783+0000 7f18a44d5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-03-24T23:47:19.800 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-03-24T23:47:19.799+0000 7ffbd8287200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-03-24T23:47:19.813 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.811+0000 7fed32da1700 1 Processor -- start 2023-03-24T23:47:19.813 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.811+0000 7fed32da1700 1 -- start start 2023-03-24T23:47:19.815 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.812+0000 7fed32da1700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed2c15c800 0x7fed2c0cd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:19.815 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.812+0000 7fed32da1700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fed2c0b5d90 0x7fed2c0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:19.816 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.812+0000 7fed32da1700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fed2c0d2410 0x7fed2c0d2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:19.816 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.812+0000 7fed32da1700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed2c15cbd0 con 0x7fed2c0d2410 2023-03-24T23:47:19.816 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.812+0000 7fed32da1700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed2c0cdc90 con 0x7fed2c0b5d90 2023-03-24T23:47:19.816 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.812+0000 7fed32da1700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c0d1650 con 0x7fed2c15c800 2023-03-24T23:47:19.816 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.812+0000 7fed3159e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fed2c0b5d90 0x7fed2c0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:19.816 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.812+0000 7fed31d9f700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed2c15c800 0x7fed2c0cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:19.816 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.812+0000 7fed3159e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fed2c0b5d90 0x7fed2c0b6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50784/0 (socket says 172.21.15.146:50784) 2023-03-24T23:47:19.817 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.812+0000 7fed31d9f700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed2c15c800 0x7fed2c0cd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.146:38058/0 (socket says 172.21.15.146:38058) 2023-03-24T23:47:19.817 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.812+0000 7fed3159e700 1 -- 172.21.15.146:0/2248587147 learned_addr learned my addr 172.21.15.146:0/2248587147 (peer_addr_for_me v2:172.21.15.146:0/0) 2023-03-24T23:47:19.817 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.812+0000 7fed325a0700 1 --2- 172.21.15.146:0/2248587147 >> v2:172.21.15.72:3300/0 conn(0x7fed2c0d2410 0x7fed2c0d2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:19.817 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.813+0000 7fed31d9f700 1 -- 172.21.15.146:0/2248587147 >> v2:172.21.15.146:3300/0 conn(0x7fed2c0b5d90 msgr2=0x7fed2c0b6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T23:47:19.817 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.813+0000 7fed31d9f700 1 --2- 172.21.15.146:0/2248587147 >> v2:172.21.15.146:3300/0 conn(0x7fed2c0b5d90 0x7fed2c0b6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:19.817 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.813+0000 7fed31d9f700 1 -- 172.21.15.146:0/2248587147 >> v2:172.21.15.72:3300/0 conn(0x7fed2c0d2410 msgr2=0x7fed2c0d2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T23:47:19.817 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.813+0000 7fed31d9f700 1 --2- 172.21.15.146:0/2248587147 >> v2:172.21.15.72:3300/0 conn(0x7fed2c0d2410 0x7fed2c0d2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:19.818 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.813+0000 7fed31d9f700 1 -- 172.21.15.146:0/2248587147 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed2c0d1930 con 0x7fed2c15c800 2023-03-24T23:47:19.818 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.813+0000 7fed3159e700 1 --2- 172.21.15.146:0/2248587147 >> v2:172.21.15.146:3300/0 conn(0x7fed2c0b5d90 0x7fed2c0b6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-24T23:47:19.818 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.813+0000 7fed31d9f700 1 --2- 172.21.15.146:0/2248587147 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed2c15c800 0x7fed2c0cd590 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fed2c0b5a40 tx=0x7fed28009b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a081fddaea9331f server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:19.818 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.813+0000 7fed30d9d700 1 -- 172.21.15.146:0/2248587147 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed28011830 con 0x7fed2c15c800 2023-03-24T23:47:19.818 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.813+0000 7fed30d9d700 1 -- 172.21.15.146:0/2248587147 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed28011990 con 0x7fed2c15c800 2023-03-24T23:47:19.818 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.813+0000 7fed30d9d700 1 -- 172.21.15.146:0/2248587147 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed28011d20 con 0x7fed2c15c800 2023-03-24T23:47:19.818 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.813+0000 7fed32da1700 1 -- 172.21.15.146:0/2248587147 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed2c15c800 msgr2=0x7fed2c0cd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:19.818 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.813+0000 7fed32da1700 1 --2- 172.21.15.146:0/2248587147 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed2c15c800 0x7fed2c0cd590 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fed2c0b5a40 tx=0x7fed28009b20 comp rx=0 tx=0).stop 2023-03-24T23:47:19.819 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.813+0000 7fed32da1700 1 -- 172.21.15.146:0/2248587147 shutdown_connections 2023-03-24T23:47:19.819 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.813+0000 7fed32da1700 1 --2- 172.21.15.146:0/2248587147 >> v2:172.21.15.72:3300/0 conn(0x7fed2c0d2410 0x7fed2c0d2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:19.819 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.813+0000 7fed32da1700 1 --2- 172.21.15.146:0/2248587147 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed2c15c800 0x7fed2c0cd590 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:19.819 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.813+0000 7fed32da1700 1 --2- 172.21.15.146:0/2248587147 >> v2:172.21.15.146:3300/0 conn(0x7fed2c0b5d90 0x7fed2c0b6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:19.819 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.813+0000 7fed32da1700 1 -- 172.21.15.146:0/2248587147 >> 172.21.15.146:0/2248587147 conn(0x7fed2c06aea0 msgr2=0x7fed2c06b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:19.819 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.814+0000 7fed32da1700 1 -- 172.21.15.146:0/2248587147 shutdown_connections 2023-03-24T23:47:19.819 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.814+0000 7fed32da1700 1 -- 172.21.15.146:0/2248587147 wait complete. 2023-03-24T23:47:19.821 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.815+0000 7fed32da1700 1 Processor -- start 2023-03-24T23:47:19.822 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.815+0000 7fed32da1700 1 -- start start 2023-03-24T23:47:19.822 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.815+0000 7fed32da1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed2c0b5d90 0x7fed2c0ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:19.822 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.816+0000 7fed32da1700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed2c0d2410 0x7fed2c15ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:19.822 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.816+0000 7fed32da1700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fed2c15c800 0x7fed2c15e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:19.822 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.816+0000 7fed32da1700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c15f730 con 0x7fed2c15c800 2023-03-24T23:47:19.822 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.816+0000 7fed32da1700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c15e840 con 0x7fed2c0b5d90 2023-03-24T23:47:19.822 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.816+0000 7fed32da1700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c15eab0 con 0x7fed2c0d2410 2023-03-24T23:47:19.823 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.816+0000 7fed31d9f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed2c0b5d90 0x7fed2c0ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:19.823 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.816+0000 7fed3159e700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed2c0d2410 0x7fed2c15ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:19.823 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.816+0000 7fed31d9f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed2c0b5d90 0x7fed2c0ccd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50800/0 (socket says 172.21.15.146:50800) 2023-03-24T23:47:19.823 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.816+0000 7fed31d9f700 1 -- 172.21.15.146:0/341264385 learned_addr learned my addr 172.21.15.146:0/341264385 (peer_addr_for_me v2:172.21.15.146:0/0) 2023-03-24T23:47:19.823 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.816+0000 7fed325a0700 1 --2- 172.21.15.146:0/341264385 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fed2c15c800 0x7fed2c15e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:19.824 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.816+0000 7fed31d9f700 1 -- 172.21.15.146:0/341264385 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed2c0d2410 msgr2=0x7fed2c15ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:19.824 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.816+0000 7fed31d9f700 1 --2- 172.21.15.146:0/341264385 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed2c0d2410 0x7fed2c15ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:19.824 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.816+0000 7fed31d9f700 1 -- 172.21.15.146:0/341264385 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fed2c15c800 msgr2=0x7fed2c15e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:19.824 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.816+0000 7fed31d9f700 1 --2- 172.21.15.146:0/341264385 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fed2c15c800 0x7fed2c15e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:19.824 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.816+0000 7fed31d9f700 1 -- 172.21.15.146:0/341264385 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed1c009550 con 0x7fed2c0b5d90 2023-03-24T23:47:19.824 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.816+0000 7fed3159e700 1 --2- 172.21.15.146:0/341264385 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed2c0d2410 0x7fed2c15ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-24T23:47:19.824 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.816+0000 7fed325a0700 1 --2- 172.21.15.146:0/341264385 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fed2c15c800 0x7fed2c15e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-24T23:47:19.824 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.816+0000 7fed31d9f700 1 --2- 172.21.15.146:0/341264385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed2c0b5d90 0x7fed2c0ccd40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fed28000c00 tx=0x7fed28004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:19.825 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.816+0000 7fed227fc700 1 -- 172.21.15.146:0/341264385 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed28004450 con 0x7fed2c0b5d90 2023-03-24T23:47:19.825 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.816+0000 7fed227fc700 1 -- 172.21.15.146:0/341264385 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed2801e070 con 0x7fed2c0b5d90 2023-03-24T23:47:19.825 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.816+0000 7fed32da1700 1 -- 172.21.15.146:0/341264385 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed280094a0 con 0x7fed2c0b5d90 2023-03-24T23:47:19.825 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.816+0000 7fed227fc700 1 -- 172.21.15.146:0/341264385 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed280038e0 con 0x7fed2c0b5d90 2023-03-24T23:47:19.825 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.816+0000 7fed32da1700 1 -- 172.21.15.146:0/341264385 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed2c0d1ef0 con 0x7fed2c0b5d90 2023-03-24T23:47:19.825 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.817+0000 7fed227fc700 1 -- 172.21.15.146:0/341264385 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fed28003d40 con 0x7fed2c0b5d90 2023-03-24T23:47:19.825 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.817+0000 7fed227fc700 1 -- 172.21.15.146:0/341264385 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fed28024400 con 0x7fed2c0b5d90 2023-03-24T23:47:19.826 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.817+0000 7fed32da1700 1 -- 172.21.15.146:0/341264385 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed2c0d1c10 con 0x7fed2c0b5d90 2023-03-24T23:47:19.826 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.821+0000 7fed227fc700 1 -- 172.21.15.146:0/341264385 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7fed28032060 con 0x7fed2c0b5d90 2023-03-24T23:47:19.866 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-24T23:47:19.865+0000 7f18a44d5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-03-24T23:47:19.891 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-03-24T23:47:19.890+0000 7ffbd8287200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-03-24T23:47:19.905 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.904+0000 7fed32da1700 1 -- 172.21.15.146:0/341264385 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd new", "uuid": "17c7d98c-55ba-4001-8eac-475f82cc31e1", "id": 3} v 0) v1 -- 0x7fed2c0b49c0 con 0x7fed2c0b5d90 2023-03-24T23:47:19.914 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.912+0000 7fed227fc700 1 -- 172.21.15.146:0/341264385 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "17c7d98c-55ba-4001-8eac-475f82cc31e1", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fed28020070 con 0x7fed2c0b5d90 2023-03-24T23:47:19.914 INFO:teuthology.orchestra.run.smithi146.stdout:3 2023-03-24T23:47:19.915 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.913+0000 7fed32da1700 1 -- 172.21.15.146:0/341264385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed2c0b5d90 msgr2=0x7fed2c0ccd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:19.915 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.913+0000 7fed32da1700 1 --2- 172.21.15.146:0/341264385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed2c0b5d90 0x7fed2c0ccd40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fed28000c00 tx=0x7fed28004750 comp rx=0 tx=0).stop 2023-03-24T23:47:19.915 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.913+0000 7fed32da1700 1 -- 172.21.15.146:0/341264385 shutdown_connections 2023-03-24T23:47:19.915 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.913+0000 7fed32da1700 1 --2- 172.21.15.146:0/341264385 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fed2c15c800 0x7fed2c15e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:19.916 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.913+0000 7fed32da1700 1 --2- 172.21.15.146:0/341264385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed2c0b5d90 0x7fed2c0ccd40 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:19.916 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.913+0000 7fed32da1700 1 --2- 172.21.15.146:0/341264385 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed2c0d2410 0x7fed2c15ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:19.916 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.913+0000 7fed32da1700 1 -- 172.21.15.146:0/341264385 >> 172.21.15.146:0/341264385 conn(0x7fed2c06aea0 msgr2=0x7fed2c0d3680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:19.916 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.913+0000 7fed32da1700 1 -- 172.21.15.146:0/341264385 shutdown_connections 2023-03-24T23:47:19.916 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:19.913+0000 7fed32da1700 1 -- 172.21.15.146:0/341264385 wait complete. 2023-03-24T23:47:19.928 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new c78725a3-3c83-4119-a657-b19b187170e3 4 2023-03-24T23:47:20.051 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-24T23:47:20.051+0000 7f18a44d5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-03-24T23:47:20.091 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-03-24T23:47:20.090+0000 7ffbd8287200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-03-24T23:47:20.093 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.092+0000 7fa91aaca700 1 Processor -- start 2023-03-24T23:47:20.093 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.092+0000 7fa91aaca700 1 -- start start 2023-03-24T23:47:20.094 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.093+0000 7fa91aaca700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa91415cc30 0x7fa91415de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:20.095 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.093+0000 7fa91aaca700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa9140cf740 0x7fa9140cdf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:20.095 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.093+0000 7fa91aaca700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9140cf2b0 con 0x7fa91406b5b0 2023-03-24T23:47:20.095 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.093+0000 7fa91aaca700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9140cfbb0 con 0x7fa9140cf740 2023-03-24T23:47:20.095 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.093+0000 7fa91aaca700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9140ce440 con 0x7fa91415cc30 2023-03-24T23:47:20.095 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.093+0000 7fa913fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa9140cf740 0x7fa9140cdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:20.095 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.093+0000 7fa918866700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa91415cc30 0x7fa91415de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:20.095 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.093+0000 7fa913fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa9140cf740 0x7fa9140cdf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50812/0 (socket says 172.21.15.146:50812) 2023-03-24T23:47:20.096 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.093+0000 7fa919067700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fa91406b5b0 0x7fa9140ca4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:49674/0 (socket says 172.21.15.146:49674) 2023-03-24T23:47:20.096 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.093+0000 7fa918866700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa91415cc30 0x7fa91415de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.146:38078/0 (socket says 172.21.15.146:38078) 2023-03-24T23:47:20.096 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.093+0000 7fa913fff700 1 -- 172.21.15.146:0/1877661 learned_addr learned my addr 172.21.15.146:0/1877661 (peer_addr_for_me v2:172.21.15.146:0/0) 2023-03-24T23:47:20.096 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.093+0000 7fa913fff700 1 -- 172.21.15.146:0/1877661 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa91415cc30 msgr2=0x7fa91415de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T23:47:20.096 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.093+0000 7fa913fff700 1 --2- 172.21.15.146:0/1877661 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa91415cc30 0x7fa91415de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:20.096 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.093+0000 7fa913fff700 1 -- 172.21.15.146:0/1877661 >> v1:172.21.15.146:6789/0 conn(0x7fa91406b5b0 legacy=0x7fa9140ca4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:20.096 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.093+0000 7fa913fff700 1 -- 172.21.15.146:0/1877661 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9140ce6a0 con 0x7fa9140cf740 2023-03-24T23:47:20.097 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.093+0000 7fa9137fe700 1 -- 172.21.15.146:0/1877661 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2654307482 0 0) 0x7fa9140cf2b0 con 0x7fa91406b5b0 2023-03-24T23:47:20.097 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.093+0000 7fa913fff700 1 --2- 172.21.15.146:0/1877661 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9140cf740 0x7fa9140cdf00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa90000bb90 tx=0x7fa90000bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3891bf41ef00cc67 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:20.097 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.093+0000 7fa9137fe700 1 -- 172.21.15.146:0/1877661 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90000dd30 con 0x7fa9140cf740 2023-03-24T23:47:20.097 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.093+0000 7fa9137fe700 1 -- 172.21.15.146:0/1877661 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90000de90 con 0x7fa9140cf740 2023-03-24T23:47:20.097 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.093+0000 7fa9137fe700 1 -- 172.21.15.146:0/1877661 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa900010690 con 0x7fa9140cf740 2023-03-24T23:47:20.099 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.094+0000 7fa91aaca700 1 -- 172.21.15.146:0/1877661 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9140cf740 msgr2=0x7fa9140cdf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:20.099 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.094+0000 7fa91aaca700 1 --2- 172.21.15.146:0/1877661 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9140cf740 0x7fa9140cdf00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa90000bb90 tx=0x7fa90000bea0 comp rx=0 tx=0).stop 2023-03-24T23:47:20.100 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.094+0000 7fa91aaca700 1 -- 172.21.15.146:0/1877661 shutdown_connections 2023-03-24T23:47:20.100 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.094+0000 7fa91aaca700 1 --2- 172.21.15.146:0/1877661 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa91415cc30 0x7fa91415de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:20.100 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.094+0000 7fa91aaca700 1 --2- 172.21.15.146:0/1877661 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9140cf740 0x7fa9140cdf00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:20.100 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.094+0000 7fa91aaca700 1 -- 172.21.15.146:0/1877661 >> 172.21.15.146:0/1877661 conn(0x7fa9140d0a00 msgr2=0x7fa9140cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:20.100 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.094+0000 7fa91aaca700 1 -- 172.21.15.146:0/1877661 shutdown_connections 2023-03-24T23:47:20.100 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.094+0000 7fa91aaca700 1 -- 172.21.15.146:0/1877661 wait complete. 2023-03-24T23:47:20.100 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.095+0000 7fa91aaca700 1 Processor -- start 2023-03-24T23:47:20.100 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.095+0000 7fa91aaca700 1 -- start start 2023-03-24T23:47:20.101 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.096+0000 7fa91aaca700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa91406b5b0 0x7fa9140ae820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:20.101 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.096+0000 7fa91aaca700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa9140cf740 0x7fa9141f2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:20.101 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.096+0000 7fa91aaca700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa91415cc30 0x7fa9141f1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:20.101 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.096+0000 7fa91aaca700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9141f3400 con 0x7fa9140cf740 2023-03-24T23:47:20.101 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.096+0000 7fa91aaca700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9140c3890 con 0x7fa91415cc30 2023-03-24T23:47:20.101 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.096+0000 7fa91aaca700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9141f2260 con 0x7fa91406b5b0 2023-03-24T23:47:20.101 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.096+0000 7fa918866700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa91406b5b0 0x7fa9140ae820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:20.102 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.096+0000 7fa919067700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa91415cc30 0x7fa9141f1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:20.102 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.096+0000 7fa918866700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa91406b5b0 0x7fa9140ae820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.146:38082/0 (socket says 172.21.15.146:38082) 2023-03-24T23:47:20.102 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.096+0000 7fa918866700 1 -- 172.21.15.146:0/3960883590 learned_addr learned my addr 172.21.15.146:0/3960883590 (peer_addr_for_me v2:172.21.15.146:0/0) 2023-03-24T23:47:20.102 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.096+0000 7fa913fff700 1 --2- 172.21.15.146:0/3960883590 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa9140cf740 0x7fa9141f2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:20.102 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.097+0000 7fa919067700 1 -- 172.21.15.146:0/3960883590 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa91406b5b0 msgr2=0x7fa9140ae820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:20.102 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.097+0000 7fa919067700 1 --2- 172.21.15.146:0/3960883590 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa91406b5b0 0x7fa9140ae820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:20.102 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.097+0000 7fa919067700 1 -- 172.21.15.146:0/3960883590 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa9140cf740 msgr2=0x7fa9141f2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:20.102 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.097+0000 7fa919067700 1 --2- 172.21.15.146:0/3960883590 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa9140cf740 0x7fa9141f2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:20.103 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.097+0000 7fa919067700 1 -- 172.21.15.146:0/3960883590 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa90000b7f0 con 0x7fa91415cc30 2023-03-24T23:47:20.103 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.097+0000 7fa918866700 1 --2- 172.21.15.146:0/3960883590 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa91406b5b0 0x7fa9140ae820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-24T23:47:20.103 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.097+0000 7fa919067700 1 --2- 172.21.15.146:0/3960883590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa91415cc30 0x7fa9141f1d20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00cbe0 tx=0x7fa90c00cef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:20.103 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.097+0000 7fa9117fa700 1 -- 172.21.15.146:0/3960883590 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c00aaa0 con 0x7fa91415cc30 2023-03-24T23:47:20.103 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.097+0000 7fa9117fa700 1 -- 172.21.15.146:0/3960883590 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90c00ac00 con 0x7fa91415cc30 2023-03-24T23:47:20.103 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.097+0000 7fa9117fa700 1 -- 172.21.15.146:0/3960883590 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c0070d0 con 0x7fa91415cc30 2023-03-24T23:47:20.103 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.097+0000 7fa91aaca700 1 -- 172.21.15.146:0/3960883590 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9140cec60 con 0x7fa91415cc30 2023-03-24T23:47:20.103 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.097+0000 7fa91aaca700 1 -- 172.21.15.146:0/3960883590 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9140ce980 con 0x7fa91415cc30 2023-03-24T23:47:20.104 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.097+0000 7fa9117fa700 1 -- 172.21.15.146:0/3960883590 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa90c007530 con 0x7fa91415cc30 2023-03-24T23:47:20.104 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.097+0000 7fa9117fa700 1 -- 172.21.15.146:0/3960883590 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fa90c012dc0 con 0x7fa91415cc30 2023-03-24T23:47:20.104 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.098+0000 7fa91aaca700 1 -- 172.21.15.146:0/3960883590 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9140cef40 con 0x7fa91415cc30 2023-03-24T23:47:20.104 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.100+0000 7fa9117fa700 1 -- 172.21.15.146:0/3960883590 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7fa90c018070 con 0x7fa91415cc30 2023-03-24T23:47:20.185 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.184+0000 7fa91aaca700 1 -- 172.21.15.146:0/3960883590 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c78725a3-3c83-4119-a657-b19b187170e3", "id": 4} v 0) v1 -- 0x7fa9140b2850 con 0x7fa91415cc30 2023-03-24T23:47:20.193 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.192+0000 7fa9117fa700 1 -- 172.21.15.146:0/3960883590 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c78725a3-3c83-4119-a657-b19b187170e3", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa90c012400 con 0x7fa91415cc30 2023-03-24T23:47:20.193 INFO:teuthology.orchestra.run.smithi146.stdout:4 2023-03-24T23:47:20.194 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.193+0000 7fa91aaca700 1 -- 172.21.15.146:0/3960883590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa91415cc30 msgr2=0x7fa9141f1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:20.195 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.193+0000 7fa91aaca700 1 --2- 172.21.15.146:0/3960883590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa91415cc30 0x7fa9141f1d20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00cbe0 tx=0x7fa90c00cef0 comp rx=0 tx=0).stop 2023-03-24T23:47:20.195 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.193+0000 7fa91aaca700 1 -- 172.21.15.146:0/3960883590 shutdown_connections 2023-03-24T23:47:20.195 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.193+0000 7fa91aaca700 1 --2- 172.21.15.146:0/3960883590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa91415cc30 0x7fa9141f1d20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:20.195 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.193+0000 7fa91aaca700 1 --2- 172.21.15.146:0/3960883590 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa91406b5b0 0x7fa9140ae820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:20.195 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.193+0000 7fa91aaca700 1 --2- 172.21.15.146:0/3960883590 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa9140cf740 0x7fa9141f2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:20.195 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.193+0000 7fa91aaca700 1 -- 172.21.15.146:0/3960883590 >> 172.21.15.146:0/3960883590 conn(0x7fa9140d0a00 msgr2=0x7fa91406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:20.195 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.193+0000 7fa91aaca700 1 -- 172.21.15.146:0/3960883590 shutdown_connections 2023-03-24T23:47:20.196 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.193+0000 7fa91aaca700 1 -- 172.21.15.146:0/3960883590 wait complete. 2023-03-24T23:47:20.206 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new 151bcb4f-de54-4e07-be8b-d212f724aec5 5 2023-03-24T23:47:20.330 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.329+0000 7f62121b0700 1 Processor -- start 2023-03-24T23:47:20.330 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.329+0000 7f62121b0700 1 -- start start 2023-03-24T23:47:20.332 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.329+0000 7f62121b0700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f620c0ce150 0x7f620c0b0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:20.332 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.329+0000 7f62121b0700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f620c0ce5c0 0x7f620c0cca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:20.332 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.329+0000 7f62121b0700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f620c0cb960 con 0x7f620c06b5b0 2023-03-24T23:47:20.333 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.329+0000 7f62121b0700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f620c0cbc40 con 0x7f620c0ce150 2023-03-24T23:47:20.333 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.329+0000 7f62121b0700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f620c0cbf20 con 0x7f620c0ce5c0 2023-03-24T23:47:20.333 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.330+0000 7f620bfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f620c0ce5c0 0x7f620c0cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:20.333 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.330+0000 7f620b7fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f620c06b5b0 0x7f620c06b980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:49682/0 (socket says 172.21.15.146:49682) 2023-03-24T23:47:20.333 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.330+0000 7f620b7fe700 1 -- 172.21.15.146:0/303214796 learned_addr learned my addr 172.21.15.146:0/303214796 (peer_addr_for_me v1:172.21.15.146:0/0) 2023-03-24T23:47:20.334 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.330+0000 7f620affd700 1 --2- 172.21.15.146:0/303214796 >> v2:172.21.15.72:3300/0 conn(0x7f620c0ce150 0x7f620c0b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:20.334 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.330+0000 7f620a7fc700 1 -- 172.21.15.146:0/303214796 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2606250093 0 0) 0x7f620c0cb960 con 0x7f620c06b5b0 2023-03-24T23:47:20.334 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.330+0000 7f620a7fc700 1 -- 172.21.15.146:0/303214796 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61f80035b0 con 0x7f620c06b5b0 2023-03-24T23:47:20.334 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.330+0000 7f620bfff700 1 -- 172.21.15.146:0/303214796 >> v2:172.21.15.72:3300/0 conn(0x7f620c0ce150 msgr2=0x7f620c0b0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T23:47:20.334 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.330+0000 7f620bfff700 1 --2- 172.21.15.146:0/303214796 >> v2:172.21.15.72:3300/0 conn(0x7f620c0ce150 0x7f620c0b0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:20.334 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.330+0000 7f620bfff700 1 -- 172.21.15.146:0/303214796 >> v1:172.21.15.146:6789/0 conn(0x7f620c06b5b0 legacy=0x7f620c06b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:20.335 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.330+0000 7f620bfff700 1 -- 172.21.15.146:0/303214796 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f620c0cc200 con 0x7f620c0ce5c0 2023-03-24T23:47:20.335 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.330+0000 7f620bfff700 1 --2- 172.21.15.146:0/303214796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f620c0ce5c0 0x7f620c0cca00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f620c0c4a80 tx=0x7f6204009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cba5c59ab232c53 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:20.335 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.330+0000 7f620a7fc700 1 -- 172.21.15.146:0/303214796 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6204011830 con 0x7f620c0ce5c0 2023-03-24T23:47:20.335 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.330+0000 7f620a7fc700 1 -- 172.21.15.146:0/303214796 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6204011990 con 0x7f620c0ce5c0 2023-03-24T23:47:20.336 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.330+0000 7f620a7fc700 1 -- 172.21.15.146:0/303214796 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6204011d20 con 0x7f620c0ce5c0 2023-03-24T23:47:20.336 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.331+0000 7f62121b0700 1 -- 172.21.15.146:0/303214796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f620c0ce5c0 msgr2=0x7f620c0cca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:20.336 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.331+0000 7f62121b0700 1 --2- 172.21.15.146:0/303214796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f620c0ce5c0 0x7f620c0cca00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f620c0c4a80 tx=0x7f6204009b20 comp rx=0 tx=0).stop 2023-03-24T23:47:20.336 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.331+0000 7f62121b0700 1 -- 172.21.15.146:0/303214796 shutdown_connections 2023-03-24T23:47:20.336 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.331+0000 7f62121b0700 1 --2- 172.21.15.146:0/303214796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f620c0ce5c0 0x7f620c0cca00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:20.336 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.331+0000 7f62121b0700 1 --2- 172.21.15.146:0/303214796 >> v2:172.21.15.72:3300/0 conn(0x7f620c0ce150 0x7f620c0b0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:20.336 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.331+0000 7f62121b0700 1 -- 172.21.15.146:0/303214796 >> 172.21.15.146:0/303214796 conn(0x7f620c09a2e0 msgr2=0x7f620c0d0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:20.337 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.331+0000 7f62121b0700 1 -- 172.21.15.146:0/303214796 shutdown_connections 2023-03-24T23:47:20.337 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.331+0000 7f62121b0700 1 -- 172.21.15.146:0/303214796 wait complete. 2023-03-24T23:47:20.339 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.333+0000 7f62121b0700 1 Processor -- start 2023-03-24T23:47:20.339 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.333+0000 7f62121b0700 1 -- start start 2023-03-24T23:47:20.339 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.334+0000 7f62121b0700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f620c06b5b0 0x7f620c0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:20.339 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.334+0000 7f62121b0700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f620c0ce150 0x7f620c0b1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:20.339 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.334+0000 7f62121b0700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f620c0ce5c0 0x7f620c0b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:20.339 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.334+0000 7f62121b0700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620c1f64e0 con 0x7f620c06b5b0 2023-03-24T23:47:20.340 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.334+0000 7f62121b0700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620c0c77f0 con 0x7f620c0ce150 2023-03-24T23:47:20.340 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.334+0000 7f62121b0700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f620c1f5bb0 con 0x7f620c0ce5c0 2023-03-24T23:47:20.340 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.334+0000 7f620affd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f620c0ce150 0x7f620c0b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:20.340 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.334+0000 7f620bfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f620c0ce5c0 0x7f620c0b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:20.340 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.334+0000 7f620affd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f620c0ce150 0x7f620c0b1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50846/0 (socket says 172.21.15.146:50846) 2023-03-24T23:47:20.340 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.335+0000 7f620affd700 1 -- 172.21.15.146:0/3871938969 learned_addr learned my addr 172.21.15.146:0/3871938969 (peer_addr_for_me v2:172.21.15.146:0/0) 2023-03-24T23:47:20.340 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.335+0000 7f620b7fe700 1 --2- 172.21.15.146:0/3871938969 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f620c06b5b0 0x7f620c0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:20.341 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.335+0000 7f620affd700 1 -- 172.21.15.146:0/3871938969 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f620c0ce5c0 msgr2=0x7f620c0b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:20.341 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.335+0000 7f620affd700 1 --2- 172.21.15.146:0/3871938969 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f620c0ce5c0 0x7f620c0b1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:20.341 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.335+0000 7f620affd700 1 -- 172.21.15.146:0/3871938969 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f620c06b5b0 msgr2=0x7f620c0ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:20.341 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.335+0000 7f620affd700 1 --2- 172.21.15.146:0/3871938969 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f620c06b5b0 0x7f620c0ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:20.341 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.335+0000 7f620affd700 1 -- 172.21.15.146:0/3871938969 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62040094a0 con 0x7f620c0ce150 2023-03-24T23:47:20.341 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.335+0000 7f620b7fe700 1 --2- 172.21.15.146:0/3871938969 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f620c06b5b0 0x7f620c0ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-24T23:47:20.341 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.335+0000 7f620bfff700 1 --2- 172.21.15.146:0/3871938969 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f620c0ce5c0 0x7f620c0b1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-24T23:47:20.341 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.335+0000 7f620affd700 1 --2- 172.21.15.146:0/3871938969 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f620c0ce150 0x7f620c0b1540 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f61f400efc0 tx=0x7f61f400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:20.342 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.335+0000 7f6203fff700 1 -- 172.21.15.146:0/3871938969 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f400cd10 con 0x7f620c0ce150 2023-03-24T23:47:20.342 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.335+0000 7f6203fff700 1 -- 172.21.15.146:0/3871938969 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f400f040 con 0x7f620c0ce150 2023-03-24T23:47:20.342 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.335+0000 7f62121b0700 1 -- 172.21.15.146:0/3871938969 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f620c0cb5f0 con 0x7f620c0ce150 2023-03-24T23:47:20.342 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.335+0000 7f6203fff700 1 -- 172.21.15.146:0/3871938969 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f40146a0 con 0x7f620c0ce150 2023-03-24T23:47:20.342 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.335+0000 7f62121b0700 1 -- 172.21.15.146:0/3871938969 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f620c0cc4e0 con 0x7f620c0ce150 2023-03-24T23:47:20.342 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.336+0000 7f6203fff700 1 -- 172.21.15.146:0/3871938969 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f61f4014b00 con 0x7f620c0ce150 2023-03-24T23:47:20.342 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.336+0000 7f6203fff700 1 -- 172.21.15.146:0/3871938969 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f61f4013960 con 0x7f620c0ce150 2023-03-24T23:47:20.342 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.336+0000 7f62121b0700 1 -- 172.21.15.146:0/3871938969 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f620c0b04e0 con 0x7f620c0ce150 2023-03-24T23:47:20.343 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.338+0000 7f6203fff700 1 -- 172.21.15.146:0/3871938969 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f61f4018020 con 0x7f620c0ce150 2023-03-24T23:47:20.423 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.422+0000 7f62121b0700 1 -- 172.21.15.146:0/3871938969 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd new", "uuid": "151bcb4f-de54-4e07-be8b-d212f724aec5", "id": 5} v 0) v1 -- 0x7f620c1f6650 con 0x7f620c0ce150 2023-03-24T23:47:20.431 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.430+0000 7f6203fff700 1 -- 172.21.15.146:0/3871938969 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "151bcb4f-de54-4e07-be8b-d212f724aec5", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f61f401d400 con 0x7f620c0ce150 2023-03-24T23:47:20.431 INFO:teuthology.orchestra.run.smithi146.stdout:5 2023-03-24T23:47:20.432 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.430+0000 7f62121b0700 1 -- 172.21.15.146:0/3871938969 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f620c0ce150 msgr2=0x7f620c0b1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:20.432 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.430+0000 7f62121b0700 1 --2- 172.21.15.146:0/3871938969 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f620c0ce150 0x7f620c0b1540 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f61f400efc0 tx=0x7f61f400c3a0 comp rx=0 tx=0).stop 2023-03-24T23:47:20.432 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.431+0000 7f62121b0700 1 -- 172.21.15.146:0/3871938969 shutdown_connections 2023-03-24T23:47:20.432 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.431+0000 7f62121b0700 1 --2- 172.21.15.146:0/3871938969 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f620c0ce5c0 0x7f620c0b1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:20.433 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.431+0000 7f62121b0700 1 --2- 172.21.15.146:0/3871938969 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f620c06b5b0 0x7f620c0ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:20.433 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.431+0000 7f62121b0700 1 --2- 172.21.15.146:0/3871938969 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f620c0ce150 0x7f620c0b1540 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:20.433 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.431+0000 7f62121b0700 1 -- 172.21.15.146:0/3871938969 >> 172.21.15.146:0/3871938969 conn(0x7f620c09a2e0 msgr2=0x7f620c15c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:20.433 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.431+0000 7f62121b0700 1 -- 172.21.15.146:0/3871938969 shutdown_connections 2023-03-24T23:47:20.433 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.431+0000 7f62121b0700 1 -- 172.21.15.146:0/3871938969 wait complete. 2023-03-24T23:47:20.443 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new e5373fa9-b440-4c05-aa55-1618cc42c18d 6 2023-03-24T23:47:20.567 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.566+0000 7fdfe428d700 1 Processor -- start 2023-03-24T23:47:20.567 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.566+0000 7fdfe428d700 1 -- start start 2023-03-24T23:47:20.569 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.567+0000 7fdfe428d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdfdc06ba10 0x7fdfdc1679e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:20.569 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.567+0000 7fdfe428d700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfdc06c4f0 con 0x7fdfdc071110 2023-03-24T23:47:20.569 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.567+0000 7fdfe428d700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfdc15df10 con 0x7fdfdc0cddc0 2023-03-24T23:47:20.569 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.567+0000 7fdfe428d700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfdc06bf30 con 0x7fdfdc06ba10 2023-03-24T23:47:20.569 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.567+0000 7fdfe282a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdfdc06ba10 0x7fdfdc1679e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:20.569 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.567+0000 7fdfe282a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdfdc06ba10 0x7fdfdc1679e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50850/0 (socket says 172.21.15.146:50850) 2023-03-24T23:47:20.570 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.567+0000 7fdfe1828700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fdfdc0cddc0 0x7fdfdc06b300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:49696/0 (socket says 172.21.15.146:49696) 2023-03-24T23:47:20.570 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.567+0000 7fdfe282a700 1 -- 172.21.15.146:0/2602725079 learned_addr learned my addr 172.21.15.146:0/2602725079 (peer_addr_for_me v2:172.21.15.146:0/0) 2023-03-24T23:47:20.570 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.567+0000 7fdfe282a700 1 -- 172.21.15.146:0/2602725079 >> v1:172.21.15.146:6789/0 conn(0x7fdfdc0cddc0 legacy=0x7fdfdc06b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:20.570 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.567+0000 7fdfe282a700 1 -- 172.21.15.146:0/2602725079 >> v1:172.21.15.72:6789/0 conn(0x7fdfdc071110 legacy=0x7fdfdc0714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:20.570 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.567+0000 7fdfe282a700 1 -- 172.21.15.146:0/2602725079 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfdc06c210 con 0x7fdfdc06ba10 2023-03-24T23:47:20.570 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.568+0000 7fdfe282a700 1 --2- 172.21.15.146:0/2602725079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdfdc06ba10 0x7fdfdc1679e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4009e50 tx=0x7fdfd400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ed94ae01c20e7b1 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:20.570 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.568+0000 7fdfe1027700 1 -- 172.21.15.146:0/2602725079 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd400e070 con 0x7fdfdc06ba10 2023-03-24T23:47:20.571 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.568+0000 7fdfe1027700 1 -- 172.21.15.146:0/2602725079 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfd400b9c0 con 0x7fdfdc06ba10 2023-03-24T23:47:20.571 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.568+0000 7fdfe1027700 1 -- 172.21.15.146:0/2602725079 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd400bd70 con 0x7fdfdc06ba10 2023-03-24T23:47:20.573 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.568+0000 7fdfe428d700 1 -- 172.21.15.146:0/2602725079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdfdc06ba10 msgr2=0x7fdfdc1679e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:20.573 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.568+0000 7fdfe428d700 1 --2- 172.21.15.146:0/2602725079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdfdc06ba10 0x7fdfdc1679e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4009e50 tx=0x7fdfd400b330 comp rx=0 tx=0).stop 2023-03-24T23:47:20.573 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.568+0000 7fdfe428d700 1 -- 172.21.15.146:0/2602725079 shutdown_connections 2023-03-24T23:47:20.573 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.568+0000 7fdfe428d700 1 --2- 172.21.15.146:0/2602725079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdfdc06ba10 0x7fdfdc1679e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:20.573 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.568+0000 7fdfe428d700 1 -- 172.21.15.146:0/2602725079 >> 172.21.15.146:0/2602725079 conn(0x7fdfdc0b2680 msgr2=0x7fdfdc0cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:20.574 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.569+0000 7fdfe428d700 1 -- 172.21.15.146:0/2602725079 shutdown_connections 2023-03-24T23:47:20.574 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.569+0000 7fdfe428d700 1 -- 172.21.15.146:0/2602725079 wait complete. 2023-03-24T23:47:20.574 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.570+0000 7fdfe428d700 1 Processor -- start 2023-03-24T23:47:20.574 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.570+0000 7fdfe428d700 1 -- start start 2023-03-24T23:47:20.574 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.570+0000 7fdfe428d700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdfdc06ba10 0x7fdfdc15c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:20.574 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.570+0000 7fdfe428d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdfdc071110 0x7fdfdc06e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:20.574 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.570+0000 7fdfe428d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdfdc0cddc0 0x7fdfdc06eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:20.574 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.570+0000 7fdfe428d700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfdc15ca60 con 0x7fdfdc0cddc0 2023-03-24T23:47:20.575 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.570+0000 7fdfe428d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfdc1e67c0 con 0x7fdfdc071110 2023-03-24T23:47:20.575 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.570+0000 7fdfe428d700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfdc1e6aa0 con 0x7fdfdc06ba10 2023-03-24T23:47:20.575 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.571+0000 7fdfe1828700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdfdc071110 0x7fdfdc06e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:20.575 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.571+0000 7fdfe1828700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdfdc071110 0x7fdfdc06e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50866/0 (socket says 172.21.15.146:50866) 2023-03-24T23:47:20.575 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.571+0000 7fdfe1828700 1 -- 172.21.15.146:0/696673335 learned_addr learned my addr 172.21.15.146:0/696673335 (peer_addr_for_me v2:172.21.15.146:0/0) 2023-03-24T23:47:20.575 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.571+0000 7fdfe2029700 1 --2- 172.21.15.146:0/696673335 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdfdc06ba10 0x7fdfdc15c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:20.575 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.571+0000 7fdfe282a700 1 --2- 172.21.15.146:0/696673335 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdfdc0cddc0 0x7fdfdc06eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:20.576 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.571+0000 7fdfe1828700 1 -- 172.21.15.146:0/696673335 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdfdc06ba10 msgr2=0x7fdfdc15c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:20.576 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.571+0000 7fdfe1828700 1 --2- 172.21.15.146:0/696673335 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdfdc06ba10 0x7fdfdc15c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:20.576 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.571+0000 7fdfe1828700 1 -- 172.21.15.146:0/696673335 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdfdc0cddc0 msgr2=0x7fdfdc06eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:20.576 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.571+0000 7fdfe1828700 1 --2- 172.21.15.146:0/696673335 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdfdc0cddc0 0x7fdfdc06eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:20.576 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.571+0000 7fdfe1828700 1 -- 172.21.15.146:0/696673335 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd4009b30 con 0x7fdfdc071110 2023-03-24T23:47:20.576 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.571+0000 7fdfe2029700 1 --2- 172.21.15.146:0/696673335 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdfdc06ba10 0x7fdfdc15c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-24T23:47:20.576 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.571+0000 7fdfe1828700 1 --2- 172.21.15.146:0/696673335 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdfdc071110 0x7fdfdc06e630 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc00c520 tx=0x7fdfcc00c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:20.576 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.571+0000 7fdfd2ffd700 1 -- 172.21.15.146:0/696673335 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc005cb0 con 0x7fdfdc071110 2023-03-24T23:47:20.577 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.571+0000 7fdfd2ffd700 1 -- 172.21.15.146:0/696673335 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfcc005e10 con 0x7fdfdc071110 2023-03-24T23:47:20.577 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.571+0000 7fdfd2ffd700 1 -- 172.21.15.146:0/696673335 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc0072d0 con 0x7fdfdc071110 2023-03-24T23:47:20.577 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.571+0000 7fdfe428d700 1 -- 172.21.15.146:0/696673335 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfdc0cae60 con 0x7fdfdc071110 2023-03-24T23:47:20.577 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.571+0000 7fdfe428d700 1 -- 172.21.15.146:0/696673335 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfdc0cab80 con 0x7fdfdc071110 2023-03-24T23:47:20.577 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.571+0000 7fdfd2ffd700 1 -- 172.21.15.146:0/696673335 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fdfcc01c440 con 0x7fdfdc071110 2023-03-24T23:47:20.577 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.572+0000 7fdfd2ffd700 1 -- 172.21.15.146:0/696673335 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fdfcc0129f0 con 0x7fdfdc071110 2023-03-24T23:47:20.578 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.572+0000 7fdfe428d700 1 -- 172.21.15.146:0/696673335 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfdc1608e0 con 0x7fdfdc071110 2023-03-24T23:47:20.578 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.574+0000 7fdfd2ffd700 1 -- 172.21.15.146:0/696673335 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7fdfcc021060 con 0x7fdfdc071110 2023-03-24T23:47:20.660 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.659+0000 7fdfe428d700 1 -- 172.21.15.146:0/696673335 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e5373fa9-b440-4c05-aa55-1618cc42c18d", "id": 6} v 0) v1 -- 0x7fdfdc09dea0 con 0x7fdfdc071110 2023-03-24T23:47:20.668 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.667+0000 7fdfd2ffd700 1 -- 172.21.15.146:0/696673335 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e5373fa9-b440-4c05-aa55-1618cc42c18d", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fdfcc017070 con 0x7fdfdc071110 2023-03-24T23:47:20.669 INFO:teuthology.orchestra.run.smithi146.stdout:6 2023-03-24T23:47:20.669 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.668+0000 7fdfe428d700 1 -- 172.21.15.146:0/696673335 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdfdc071110 msgr2=0x7fdfdc06e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:20.670 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.668+0000 7fdfe428d700 1 --2- 172.21.15.146:0/696673335 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdfdc071110 0x7fdfdc06e630 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc00c520 tx=0x7fdfcc00c830 comp rx=0 tx=0).stop 2023-03-24T23:47:20.670 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.668+0000 7fdfe428d700 1 -- 172.21.15.146:0/696673335 shutdown_connections 2023-03-24T23:47:20.670 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.668+0000 7fdfe428d700 1 --2- 172.21.15.146:0/696673335 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdfdc0cddc0 0x7fdfdc06eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:20.670 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.668+0000 7fdfe428d700 1 --2- 172.21.15.146:0/696673335 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdfdc06ba10 0x7fdfdc15c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:20.670 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.668+0000 7fdfe428d700 1 --2- 172.21.15.146:0/696673335 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdfdc071110 0x7fdfdc06e630 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:20.670 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.668+0000 7fdfe428d700 1 -- 172.21.15.146:0/696673335 >> 172.21.15.146:0/696673335 conn(0x7fdfdc0b2680 msgr2=0x7fdfdc0cbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:20.670 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.668+0000 7fdfe428d700 1 -- 172.21.15.146:0/696673335 shutdown_connections 2023-03-24T23:47:20.671 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.668+0000 7fdfe428d700 1 -- 172.21.15.146:0/696673335 wait complete. 2023-03-24T23:47:20.681 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new 1a67b0d5-8adc-4dd7-825c-6f26334a9a6f 7 2023-03-24T23:47:20.824 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-24T23:47:20.824+0000 7f18a44d5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-03-24T23:47:20.826 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.825+0000 7f909be7c700 1 Processor -- start 2023-03-24T23:47:20.826 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.825+0000 7f909be7c700 1 -- start start 2023-03-24T23:47:20.827 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.826+0000 7f909be7c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f909415ad10 0x7f90940c6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:20.827 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.826+0000 7f909be7c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f90940b4e30 0x7f909406afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:20.827 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.826+0000 7f909be7c700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9094066140 con 0x7f90940cbb70 2023-03-24T23:47:20.828 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.826+0000 7f909be7c700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90940b55c0 con 0x7f90940b4e30 2023-03-24T23:47:20.828 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.826+0000 7f909be7c700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f909415b0e0 con 0x7f909415ad10 2023-03-24T23:47:20.828 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.826+0000 7f9099417700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f90940cbb70 0x7f90940b4720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:49704/0 (socket says 172.21.15.146:49704) 2023-03-24T23:47:20.828 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.826+0000 7f9099c18700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f909415ad10 0x7f90940c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:20.828 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.826+0000 7f9099417700 1 -- 172.21.15.146:0/2245117577 learned_addr learned my addr 172.21.15.146:0/2245117577 (peer_addr_for_me v1:172.21.15.146:0/0) 2023-03-24T23:47:20.828 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.826+0000 7f909a419700 1 --2- 172.21.15.146:0/2245117577 >> v2:172.21.15.146:3300/0 conn(0x7f90940b4e30 0x7f909406afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:20.830 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.827+0000 7f9098c16700 1 -- 172.21.15.146:0/2245117577 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1117503896 0 0) 0x7f9094066140 con 0x7f90940cbb70 2023-03-24T23:47:20.830 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.827+0000 7f9098c16700 1 -- 172.21.15.146:0/2245117577 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90800035b0 con 0x7f90940cbb70 2023-03-24T23:47:20.830 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.827+0000 7f9099c18700 1 -- 172.21.15.146:0/2245117577 >> v2:172.21.15.146:3300/0 conn(0x7f90940b4e30 msgr2=0x7f909406afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T23:47:20.830 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.827+0000 7f9099c18700 1 --2- 172.21.15.146:0/2245117577 >> v2:172.21.15.146:3300/0 conn(0x7f90940b4e30 0x7f909406afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:20.830 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.827+0000 7f9099c18700 1 -- 172.21.15.146:0/2245117577 >> v1:172.21.15.146:6789/0 conn(0x7f90940cbb70 legacy=0x7f90940b4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:20.830 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.827+0000 7f9099c18700 1 -- 172.21.15.146:0/2245117577 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9094066390 con 0x7f909415ad10 2023-03-24T23:47:20.830 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.827+0000 7f909a419700 1 --2- 172.21.15.146:0/2245117577 >> v2:172.21.15.146:3300/0 conn(0x7f90940b4e30 0x7f909406afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-24T23:47:20.831 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.827+0000 7f9099c18700 1 --2- 172.21.15.146:0/2245117577 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f909415ad10 0x7f90940c6670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9090009840 tx=0x7f9090009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af9069c1a5df57b5 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:20.831 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.827+0000 7f9098c16700 1 -- 172.21.15.146:0/2245117577 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9090011830 con 0x7f909415ad10 2023-03-24T23:47:20.831 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.827+0000 7f9098c16700 1 -- 172.21.15.146:0/2245117577 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9090011990 con 0x7f909415ad10 2023-03-24T23:47:20.831 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.827+0000 7f9098c16700 1 -- 172.21.15.146:0/2245117577 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9090011d20 con 0x7f909415ad10 2023-03-24T23:47:20.831 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.827+0000 7f909be7c700 1 -- 172.21.15.146:0/2245117577 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f909415ad10 msgr2=0x7f90940c6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:20.831 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.827+0000 7f909be7c700 1 --2- 172.21.15.146:0/2245117577 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f909415ad10 0x7f90940c6670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9090009840 tx=0x7f9090009b50 comp rx=0 tx=0).stop 2023-03-24T23:47:20.831 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.828+0000 7f909be7c700 1 -- 172.21.15.146:0/2245117577 shutdown_connections 2023-03-24T23:47:20.831 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.828+0000 7f909be7c700 1 --2- 172.21.15.146:0/2245117577 >> v2:172.21.15.146:3300/0 conn(0x7f90940b4e30 0x7f909406afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:20.832 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.828+0000 7f909be7c700 1 --2- 172.21.15.146:0/2245117577 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f909415ad10 0x7f90940c6670 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:20.832 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.828+0000 7f909be7c700 1 -- 172.21.15.146:0/2245117577 >> 172.21.15.146:0/2245117577 conn(0x7f909406d5a0 msgr2=0x7f90940cf5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:20.832 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.828+0000 7f909be7c700 1 -- 172.21.15.146:0/2245117577 shutdown_connections 2023-03-24T23:47:20.832 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.828+0000 7f909be7c700 1 -- 172.21.15.146:0/2245117577 wait complete. 2023-03-24T23:47:20.833 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.829+0000 7f909be7c700 1 Processor -- start 2023-03-24T23:47:20.834 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.829+0000 7f909be7c700 1 -- start start 2023-03-24T23:47:20.834 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.830+0000 7f909be7c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90940b4e30 0x7f90940c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:20.834 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.830+0000 7f909be7c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f90940cbb70 0x7f90940c4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:20.834 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.830+0000 7f909be7c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f909415ad10 0x7f90940c4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:20.834 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.830+0000 7f909be7c700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90941f1770 con 0x7f90940cbb70 2023-03-24T23:47:20.834 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.830+0000 7f909be7c700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90941f1490 con 0x7f909415ad10 2023-03-24T23:47:20.834 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.830+0000 7f909be7c700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90941f1ac0 con 0x7f90940b4e30 2023-03-24T23:47:20.835 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.830+0000 7f9099c18700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90940b4e30 0x7f90940c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:20.835 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.830+0000 7f9099c18700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90940b4e30 0x7f90940c8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.146:38132/0 (socket says 172.21.15.146:38132) 2023-03-24T23:47:20.835 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.830+0000 7f9099c18700 1 -- 172.21.15.146:0/2077024037 learned_addr learned my addr 172.21.15.146:0/2077024037 (peer_addr_for_me v2:172.21.15.146:0/0) 2023-03-24T23:47:20.835 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.830+0000 7f909a419700 1 --2- 172.21.15.146:0/2077024037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f909415ad10 0x7f90940c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:20.835 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.830+0000 7f9099417700 1 --2- 172.21.15.146:0/2077024037 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f90940cbb70 0x7f90940c4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:20.835 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.830+0000 7f9099c18700 1 -- 172.21.15.146:0/2077024037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f909415ad10 msgr2=0x7f90940c4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:20.835 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.830+0000 7f9099c18700 1 --2- 172.21.15.146:0/2077024037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f909415ad10 0x7f90940c4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:20.835 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.830+0000 7f9099c18700 1 -- 172.21.15.146:0/2077024037 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f90940cbb70 msgr2=0x7f90940c4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:20.836 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.831+0000 7f9099c18700 1 --2- 172.21.15.146:0/2077024037 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f90940cbb70 0x7f90940c4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:20.836 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.831+0000 7f9099c18700 1 -- 172.21.15.146:0/2077024037 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f908c009a70 con 0x7f90940b4e30 2023-03-24T23:47:20.836 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.831+0000 7f9099c18700 1 --2- 172.21.15.146:0/2077024037 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90940b4e30 0x7f90940c8ce0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9090009fd0 tx=0x7f9090004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:20.836 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.831+0000 7f908a7fc700 1 -- 172.21.15.146:0/2077024037 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90900045d0 con 0x7f90940b4e30 2023-03-24T23:47:20.836 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.831+0000 7f908a7fc700 1 -- 172.21.15.146:0/2077024037 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f909001e070 con 0x7f90940b4e30 2023-03-24T23:47:20.836 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.831+0000 7f908a7fc700 1 -- 172.21.15.146:0/2077024037 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90900038e0 con 0x7f90940b4e30 2023-03-24T23:47:20.836 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.831+0000 7f909be7c700 1 -- 172.21.15.146:0/2077024037 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90900094a0 con 0x7f90940b4e30 2023-03-24T23:47:20.837 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.831+0000 7f909be7c700 1 -- 172.21.15.146:0/2077024037 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f909406a7d0 con 0x7f90940b4e30 2023-03-24T23:47:20.837 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.831+0000 7f908a7fc700 1 -- 172.21.15.146:0/2077024037 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9090003d40 con 0x7f90940b4e30 2023-03-24T23:47:20.837 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.831+0000 7f908a7fc700 1 -- 172.21.15.146:0/2077024037 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f909001aa80 con 0x7f90940b4e30 2023-03-24T23:47:20.837 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.832+0000 7f909be7c700 1 -- 172.21.15.146:0/2077024037 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f909406aab0 con 0x7f90940b4e30 2023-03-24T23:47:20.837 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.834+0000 7f908a7fc700 1 -- 172.21.15.146:0/2077024037 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f9090020070 con 0x7f90940b4e30 2023-03-24T23:47:20.915 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-03-24T23:47:20.914+0000 7ffbd8287200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-03-24T23:47:20.918 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.918+0000 7f909be7c700 1 -- 172.21.15.146:0/2077024037 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "osd new", "uuid": "1a67b0d5-8adc-4dd7-825c-6f26334a9a6f", "id": 7} v 0) v1 -- 0x7f90941f1920 con 0x7f90940b4e30 2023-03-24T23:47:20.922 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.921+0000 7f908a7fc700 1 -- 172.21.15.146:0/2077024037 <== mon.2 v2:172.21.15.146:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1a67b0d5-8adc-4dd7-825c-6f26334a9a6f", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f90941f1920 con 0x7f90940b4e30 2023-03-24T23:47:20.922 INFO:teuthology.orchestra.run.smithi146.stdout:7 2023-03-24T23:47:20.923 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.922+0000 7f909be7c700 1 -- 172.21.15.146:0/2077024037 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90940b4e30 msgr2=0x7f90940c8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:20.924 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.922+0000 7f909be7c700 1 --2- 172.21.15.146:0/2077024037 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90940b4e30 0x7f90940c8ce0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9090009fd0 tx=0x7f9090004750 comp rx=0 tx=0).stop 2023-03-24T23:47:20.924 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.922+0000 7f909be7c700 1 -- 172.21.15.146:0/2077024037 shutdown_connections 2023-03-24T23:47:20.924 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.922+0000 7f909be7c700 1 --2- 172.21.15.146:0/2077024037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f909415ad10 0x7f90940c4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:20.924 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.922+0000 7f909be7c700 1 --2- 172.21.15.146:0/2077024037 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90940b4e30 0x7f90940c8ce0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:20.924 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.922+0000 7f909be7c700 1 --2- 172.21.15.146:0/2077024037 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f90940cbb70 0x7f90940c4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:20.924 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.922+0000 7f909be7c700 1 -- 172.21.15.146:0/2077024037 >> 172.21.15.146:0/2077024037 conn(0x7f909406d5a0 msgr2=0x7f909406aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:20.924 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.922+0000 7f909be7c700 1 -- 172.21.15.146:0/2077024037 shutdown_connections 2023-03-24T23:47:20.925 INFO:teuthology.orchestra.run.smithi146.stderr:2023-03-24T23:47:20.922+0000 7f909be7c700 1 -- 172.21.15.146:0/2077024037 wait complete. 2023-03-24T23:47:20.935 INFO:tasks.ceph.osd.0:Restarting daemon 2023-03-24T23:47:20.935 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-03-24T23:47:20.937 INFO:tasks.ceph.osd.0:Started 2023-03-24T23:47:20.938 INFO:tasks.ceph.osd.1:Restarting daemon 2023-03-24T23:47:20.938 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-03-24T23:47:20.939 INFO:tasks.ceph.osd.1:Started 2023-03-24T23:47:20.940 INFO:tasks.ceph.osd.2:Restarting daemon 2023-03-24T23:47:20.940 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-03-24T23:47:20.942 INFO:tasks.ceph.osd.2:Started 2023-03-24T23:47:20.942 INFO:tasks.ceph.osd.3:Restarting daemon 2023-03-24T23:47:20.942 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-03-24T23:47:20.944 INFO:tasks.ceph.osd.3:Started 2023-03-24T23:47:20.944 INFO:tasks.ceph.osd.4:Restarting daemon 2023-03-24T23:47:20.945 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-03-24T23:47:20.946 INFO:tasks.ceph.osd.4:Started 2023-03-24T23:47:20.947 INFO:tasks.ceph.osd.5:Restarting daemon 2023-03-24T23:47:20.947 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-03-24T23:47:20.949 INFO:tasks.ceph.osd.5:Started 2023-03-24T23:47:20.949 INFO:tasks.ceph.osd.6:Restarting daemon 2023-03-24T23:47:20.949 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-03-24T23:47:20.951 INFO:tasks.ceph.osd.6:Started 2023-03-24T23:47:20.951 INFO:tasks.ceph.osd.7:Restarting daemon 2023-03-24T23:47:20.952 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-03-24T23:47:20.953 INFO:tasks.ceph.osd.7:Started 2023-03-24T23:47:20.954 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-24T23:47:20.983 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-24T23:47:20.983+0000 7f18a44d5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-03-24T23:47:21.089 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-03-24T23:47:21.088+0000 7ffbd8287200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-03-24T23:47:21.102 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.101+0000 7f937641c700 1 Processor -- start 2023-03-24T23:47:21.102 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.102+0000 7f937641c700 1 -- start start 2023-03-24T23:47:21.103 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.102+0000 7f937641c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9370165a70 0x7f9370167e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:21.103 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.102+0000 7f937641c700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f937006f1a0 con 0x7f9370071130 2023-03-24T23:47:21.103 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.102+0000 7f937641c700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f937006c4e0 con 0x7f9370161360 2023-03-24T23:47:21.104 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.102+0000 7f937641c700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9370160f40 con 0x7f9370165a70 2023-03-24T23:47:21.104 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.102+0000 7f936ffff700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f9370071130 0x7f9370071500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:52776/0 (socket says 172.21.15.72:52776) 2023-03-24T23:47:21.104 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.102+0000 7f936ffff700 1 -- 172.21.15.72:0/3711116892 learned_addr learned my addr 172.21.15.72:0/3711116892 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-03-24T23:47:21.104 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.103+0000 7f93749b9700 1 --2- 172.21.15.72:0/3711116892 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9370165a70 0x7f9370167e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:21.104 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.103+0000 7f936effd700 1 -- 172.21.15.72:0/3711116892 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3488354392 0 0) 0x7f937006f1a0 con 0x7f9370071130 2023-03-24T23:47:21.104 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.103+0000 7f936effd700 1 -- 172.21.15.72:0/3711116892 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93580035b0 con 0x7f9370071130 2023-03-24T23:47:21.104 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.103+0000 7f936effd700 1 -- 172.21.15.72:0/3711116892 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1761735345 0 0) 0x7f937006c4e0 con 0x7f9370161360 2023-03-24T23:47:21.105 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.103+0000 7f936effd700 1 -- 172.21.15.72:0/3711116892 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f937006f1a0 con 0x7f9370161360 2023-03-24T23:47:21.105 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.103+0000 7f93749b9700 1 -- 172.21.15.72:0/3711116892 >> v1:172.21.15.146:6789/0 conn(0x7f9370161360 legacy=0x7f937006b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:21.105 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.103+0000 7f93749b9700 1 -- 172.21.15.72:0/3711116892 >> v1:172.21.15.72:6789/0 conn(0x7f9370071130 legacy=0x7f9370071500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:21.105 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.103+0000 7f93749b9700 1 -- 172.21.15.72:0/3711116892 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f937015e0d0 con 0x7f9370165a70 2023-03-24T23:47:21.105 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.103+0000 7f93749b9700 1 --2- 172.21.15.72:0/3711116892 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9370165a70 0x7f9370167e50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9368009dc0 tx=0x7f936800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ee446d1de3aa21a server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:21.105 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.103+0000 7f936effd700 1 -- 172.21.15.72:0/3711116892 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936800e070 con 0x7f9370165a70 2023-03-24T23:47:21.105 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.103+0000 7f936effd700 1 -- 172.21.15.72:0/3711116892 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f936800bb20 con 0x7f9370165a70 2023-03-24T23:47:21.106 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.103+0000 7f936effd700 1 -- 172.21.15.72:0/3711116892 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9368012430 con 0x7f9370165a70 2023-03-24T23:47:21.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.104+0000 7f937641c700 1 -- 172.21.15.72:0/3711116892 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9370165a70 msgr2=0x7f9370167e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:21.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.104+0000 7f937641c700 1 --2- 172.21.15.72:0/3711116892 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9370165a70 0x7f9370167e50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9368009dc0 tx=0x7f936800b330 comp rx=0 tx=0).stop 2023-03-24T23:47:21.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.104+0000 7f937641c700 1 -- 172.21.15.72:0/3711116892 shutdown_connections 2023-03-24T23:47:21.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.104+0000 7f937641c700 1 --2- 172.21.15.72:0/3711116892 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9370165a70 0x7f9370167e50 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:21.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.104+0000 7f937641c700 1 -- 172.21.15.72:0/3711116892 >> 172.21.15.72:0/3711116892 conn(0x7f93700cfe80 msgr2=0x7f93700d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:21.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.104+0000 7f937641c700 1 -- 172.21.15.72:0/3711116892 shutdown_connections 2023-03-24T23:47:21.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.104+0000 7f937641c700 1 -- 172.21.15.72:0/3711116892 wait complete. 2023-03-24T23:47:21.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.105+0000 7f937641c700 1 Processor -- start 2023-03-24T23:47:21.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.106+0000 7f937641c700 1 -- start start 2023-03-24T23:47:21.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.106+0000 7f937641c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9370071130 0x7f9370148ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:21.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.106+0000 7f937641c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9370161360 0x7f93700ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:21.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.106+0000 7f937641c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9370165a70 0x7f93700c4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:21.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.106+0000 7f937641c700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93700c39b0 con 0x7f9370165a70 2023-03-24T23:47:21.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.106+0000 7f937641c700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93701582f0 con 0x7f9370071130 2023-03-24T23:47:21.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.106+0000 7f937641c700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93701585d0 con 0x7f9370161360 2023-03-24T23:47:21.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.106+0000 7f93749b9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9370165a70 0x7f93700c4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:21.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.106+0000 7f93749b9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9370165a70 0x7f93700c4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56102/0 (socket says 172.21.15.72:56102) 2023-03-24T23:47:21.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.106+0000 7f93749b9700 1 -- 172.21.15.72:0/3374624068 learned_addr learned my addr 172.21.15.72:0/3374624068 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-03-24T23:47:21.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.106+0000 7f936ffff700 1 --2- 172.21.15.72:0/3374624068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9370071130 0x7f9370148ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:21.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.106+0000 7f936f7fe700 1 --2- 172.21.15.72:0/3374624068 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9370161360 0x7f93700ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:21.111 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.107+0000 7f93749b9700 1 -- 172.21.15.72:0/3374624068 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9370161360 msgr2=0x7f93700ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:21.111 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.107+0000 7f93749b9700 1 --2- 172.21.15.72:0/3374624068 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9370161360 0x7f93700ac880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:21.111 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.107+0000 7f93749b9700 1 -- 172.21.15.72:0/3374624068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9370071130 msgr2=0x7f9370148ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:21.111 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.107+0000 7f93749b9700 1 --2- 172.21.15.72:0/3374624068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9370071130 0x7f9370148ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:21.111 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.107+0000 7f93749b9700 1 -- 172.21.15.72:0/3374624068 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9368009a70 con 0x7f9370165a70 2023-03-24T23:47:21.111 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.107+0000 7f936ffff700 1 --2- 172.21.15.72:0/3374624068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9370071130 0x7f9370148ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-24T23:47:21.111 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.107+0000 7f93749b9700 1 --2- 172.21.15.72:0/3374624068 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9370165a70 0x7f93700c4400 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f936800b370 tx=0x7f9368004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:21.111 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.107+0000 7f936cff9700 1 -- 172.21.15.72:0/3374624068 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936801b810 con 0x7f9370165a70 2023-03-24T23:47:21.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.107+0000 7f936cff9700 1 -- 172.21.15.72:0/3374624068 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9368020070 con 0x7f9370165a70 2023-03-24T23:47:21.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.107+0000 7f937641c700 1 -- 172.21.15.72:0/3374624068 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9370160880 con 0x7f9370165a70 2023-03-24T23:47:21.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.107+0000 7f936cff9700 1 -- 172.21.15.72:0/3374624068 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936801bbd0 con 0x7f9370165a70 2023-03-24T23:47:21.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.107+0000 7f937641c700 1 -- 172.21.15.72:0/3374624068 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f937006c200 con 0x7f9370165a70 2023-03-24T23:47:21.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.107+0000 7f936cff9700 1 -- 172.21.15.72:0/3374624068 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9368027780 con 0x7f9370165a70 2023-03-24T23:47:21.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.107+0000 7f936cff9700 1 -- 172.21.15.72:0/3374624068 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f9368022070 con 0x7f9370165a70 2023-03-24T23:47:21.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.108+0000 7f937641c700 1 -- 172.21.15.72:0/3374624068 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9370160b60 con 0x7f9370165a70 2023-03-24T23:47:21.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.112+0000 7f936cff9700 1 -- 172.21.15.72:0/3374624068 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f936801c400 con 0x7f9370165a70 2023-03-24T23:47:21.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.181+0000 7f937641c700 1 -- 172.21.15.72:0/3374624068 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9370072220 con 0x7f9370165a70 2023-03-24T23:47:21.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.182+0000 7f936cff9700 1 -- 172.21.15.72:0/3374624068 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f936801c5a0 con 0x7f9370165a70 2023-03-24T23:47:21.182 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:47:21.182 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":10,"fsid":"d3c122b7-f0b4-4c96-8f85-86f129e51880","created":"2023-03-24T23:47:18.348997+0000","modified":"2023-03-24T23:47:20.920082+0000","last_up_change":"0.000000","last_in_change":"2023-03-24T23:47:20.920082+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"993abb78-a510-4d76-af32-9015a2b3e5d2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ddfa95f0-f08c-4dce-b961-2e3a9f4bcb3d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"cfcf995d-5a1f-47c1-9691-858f305b1f47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"17c7d98c-55ba-4001-8eac-475f82cc31e1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"c78725a3-3c83-4119-a657-b19b187170e3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"151bcb4f-de54-4e07-be8b-d212f724aec5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e5373fa9-b440-4c05-aa55-1618cc42c18d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"1a67b0d5-8adc-4dd7-825c-6f26334a9a6f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-24T23:47:21.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.183+0000 7f937641c700 1 -- 172.21.15.72:0/3374624068 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9370165a70 msgr2=0x7f93700c4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:21.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.183+0000 7f937641c700 1 --2- 172.21.15.72:0/3374624068 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9370165a70 0x7f93700c4400 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f936800b370 tx=0x7f9368004ad0 comp rx=0 tx=0).stop 2023-03-24T23:47:21.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.183+0000 7f937641c700 1 -- 172.21.15.72:0/3374624068 shutdown_connections 2023-03-24T23:47:21.184 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.183+0000 7f937641c700 1 --2- 172.21.15.72:0/3374624068 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9370165a70 0x7f93700c4400 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:21.184 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.183+0000 7f937641c700 1 --2- 172.21.15.72:0/3374624068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9370071130 0x7f9370148ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:21.184 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.183+0000 7f937641c700 1 --2- 172.21.15.72:0/3374624068 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9370161360 0x7f93700ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:21.184 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.183+0000 7f937641c700 1 -- 172.21.15.72:0/3374624068 >> 172.21.15.72:0/3374624068 conn(0x7f93700cfe80 msgr2=0x7f937015a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:21.184 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.183+0000 7f937641c700 1 -- 172.21.15.72:0/3374624068 shutdown_connections 2023-03-24T23:47:21.184 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:21.183+0000 7f937641c700 1 -- 172.21.15.72:0/3374624068 wait complete. 2023-03-24T23:47:21.195 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-03-24T23:47:21.195 INFO:tasks.ceph:Waiting for OSDs to come up 2023-03-24T23:47:21.233 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-24T23:47:21.233+0000 7f18a44d5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-03-24T23:47:21.305 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-24T23:47:21.305+0000 7f18a44d5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-03-24T23:47:21.315 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-03-24T23:47:21.314+0000 7ffbd8287200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-03-24T23:47:21.391 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-03-24T23:47:21.389+0000 7ffbd8287200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-03-24T23:47:21.443 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-24T23:47:21.443+0000 7f18a44d5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-03-24T23:47:21.530 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-03-24T23:47:21.529+0000 7ffbd8287200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-03-24T23:47:21.553 INFO:tasks.ceph.osd.5.smithi146.stderr:2023-03-24T23:47:21.552+0000 7f69fb8b8640 -1 Falling back to public interface 2023-03-24T23:47:21.555 INFO:tasks.ceph.osd.1.smithi072.stderr:2023-03-24T23:47:21.555+0000 7fa92b47b640 -1 Falling back to public interface 2023-03-24T23:47:21.557 INFO:tasks.ceph.osd.6.smithi146.stderr:2023-03-24T23:47:21.556+0000 7f3c660ab640 -1 Falling back to public interface 2023-03-24T23:47:21.563 INFO:tasks.ceph.osd.0.smithi072.stderr:2023-03-24T23:47:21.563+0000 7f78af348640 -1 Falling back to public interface 2023-03-24T23:47:21.571 INFO:tasks.ceph.osd.2.smithi072.stderr:2023-03-24T23:47:21.572+0000 7fc191667640 -1 Falling back to public interface 2023-03-24T23:47:21.572 INFO:tasks.ceph.osd.3.smithi072.stderr:2023-03-24T23:47:21.572+0000 7f9e50761640 -1 Falling back to public interface 2023-03-24T23:47:21.572 INFO:tasks.ceph.osd.3.smithi072.stderr:2023-03-24T23:47:21.572+0000 7f9e47b0e700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6816/0: (98) Address already in use 2023-03-24T23:47:21.581 INFO:tasks.ceph.osd.4.smithi146.stderr:2023-03-24T23:47:21.580+0000 7f6ccce2c640 -1 Falling back to public interface 2023-03-24T23:47:21.587 INFO:tasks.ceph.osd.7.smithi146.stderr:2023-03-24T23:47:21.587+0000 7f88973dc640 -1 Falling back to public interface 2023-03-24T23:47:21.818 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-24T23:47:21.818+0000 7f18a44d5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-03-24T23:47:21.894 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-24T23:47:21.894+0000 7f18a44d5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-03-24T23:47:21.914 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-03-24T23:47:21.912+0000 7ffbd8287200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-03-24T23:47:21.988 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-03-24T23:47:21.987+0000 7ffbd8287200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-03-24T23:47:21.999 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-03-24T23:47:22.034 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-24T23:47:22.034+0000 7f18a44d5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-03-24T23:47:22.114 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.114+0000 7f392dc50700 1 Processor -- start 2023-03-24T23:47:22.114 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.114+0000 7f392dc50700 1 -- start start 2023-03-24T23:47:22.115 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.114+0000 7f392dc50700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3928159990 0x7f392806b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:22.116 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.114+0000 7f392dc50700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f392815e280 con 0x7f39280cec60 2023-03-24T23:47:22.116 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.114+0000 7f392dc50700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39280cf100 con 0x7f392806d200 2023-03-24T23:47:22.116 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.114+0000 7f392dc50700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39280d0a90 con 0x7f3928159990 2023-03-24T23:47:22.116 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.115+0000 7f3926ffd700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f39280cec60 0x7f392815db70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:52782/0 (socket says 172.21.15.72:52782) 2023-03-24T23:47:22.116 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.115+0000 7f3926ffd700 1 -- 172.21.15.72:0/4196573727 learned_addr learned my addr 172.21.15.72:0/4196573727 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-03-24T23:47:22.116 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.115+0000 7f39277fe700 1 --2- 172.21.15.72:0/4196573727 >> v2:172.21.15.146:3300/0 conn(0x7f3928159990 0x7f392806b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:22.116 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.115+0000 7f39267fc700 1 -- 172.21.15.72:0/4196573727 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4040270490 0 0) 0x7f392815e280 con 0x7f39280cec60 2023-03-24T23:47:22.116 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.115+0000 7f39267fc700 1 -- 172.21.15.72:0/4196573727 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39100035b0 con 0x7f39280cec60 2023-03-24T23:47:22.117 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.115+0000 7f39277fe700 1 -- 172.21.15.72:0/4196573727 >> v1:172.21.15.146:6789/0 conn(0x7f392806d200 legacy=0x7f392806d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:22.117 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.115+0000 7f39277fe700 1 -- 172.21.15.72:0/4196573727 >> v1:172.21.15.72:6789/0 conn(0x7f39280cec60 legacy=0x7f392815db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:22.117 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.115+0000 7f39277fe700 1 -- 172.21.15.72:0/4196573727 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f392806bb90 con 0x7f3928159990 2023-03-24T23:47:22.117 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.115+0000 7f39277fe700 1 --2- 172.21.15.72:0/4196573727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3928159990 0x7f392806b5b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3918009840 tx=0x7f3918009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1069b618d6fd9c server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:22.117 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.115+0000 7f39267fc700 1 -- 172.21.15.72:0/4196573727 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3918011830 con 0x7f3928159990 2023-03-24T23:47:22.117 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.115+0000 7f39267fc700 1 -- 172.21.15.72:0/4196573727 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3918011990 con 0x7f3928159990 2023-03-24T23:47:22.117 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.115+0000 7f39267fc700 1 -- 172.21.15.72:0/4196573727 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3918011d20 con 0x7f3928159990 2023-03-24T23:47:22.120 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.116+0000 7f392dc50700 1 -- 172.21.15.72:0/4196573727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3928159990 msgr2=0x7f392806b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:22.120 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.116+0000 7f392dc50700 1 --2- 172.21.15.72:0/4196573727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3928159990 0x7f392806b5b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3918009840 tx=0x7f3918009b50 comp rx=0 tx=0).stop 2023-03-24T23:47:22.120 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.116+0000 7f392dc50700 1 -- 172.21.15.72:0/4196573727 shutdown_connections 2023-03-24T23:47:22.120 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.116+0000 7f392dc50700 1 --2- 172.21.15.72:0/4196573727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3928159990 0x7f392806b5b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:22.120 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.116+0000 7f392dc50700 1 -- 172.21.15.72:0/4196573727 >> 172.21.15.72:0/4196573727 conn(0x7f39280cd590 msgr2=0x7f39280cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:22.120 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.116+0000 7f392dc50700 1 -- 172.21.15.72:0/4196573727 shutdown_connections 2023-03-24T23:47:22.121 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.116+0000 7f392dc50700 1 -- 172.21.15.72:0/4196573727 wait complete. 2023-03-24T23:47:22.121 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.117+0000 7f392dc50700 1 Processor -- start 2023-03-24T23:47:22.121 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.117+0000 7f392dc50700 1 -- start start 2023-03-24T23:47:22.121 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.118+0000 7f392dc50700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f392806d200 0x7f392814b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:22.121 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.118+0000 7f392dc50700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f39280cec60 0x7f39280b5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:22.121 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.118+0000 7f392dc50700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3928159990 0x7f39280b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:22.121 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.118+0000 7f392dc50700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39280c4310 con 0x7f3928159990 2023-03-24T23:47:22.121 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.118+0000 7f392dc50700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39280c8eb0 con 0x7f392806d200 2023-03-24T23:47:22.122 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.118+0000 7f392dc50700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39280c9190 con 0x7f39280cec60 2023-03-24T23:47:22.122 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.118+0000 7f3927fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3928159990 0x7f39280b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:22.122 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.118+0000 7f3927fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3928159990 0x7f39280b2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56110/0 (socket says 172.21.15.72:56110) 2023-03-24T23:47:22.122 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.118+0000 7f3927fff700 1 -- 172.21.15.72:0/129286761 learned_addr learned my addr 172.21.15.72:0/129286761 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-03-24T23:47:22.122 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.118+0000 7f39277fe700 1 --2- 172.21.15.72:0/129286761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f392806d200 0x7f392814b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:22.122 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.118+0000 7f3926ffd700 1 --2- 172.21.15.72:0/129286761 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f39280cec60 0x7f39280b5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:22.122 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.118+0000 7f3927fff700 1 -- 172.21.15.72:0/129286761 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f39280cec60 msgr2=0x7f39280b5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:22.123 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.118+0000 7f3927fff700 1 --2- 172.21.15.72:0/129286761 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f39280cec60 0x7f39280b5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:22.123 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.118+0000 7f3927fff700 1 -- 172.21.15.72:0/129286761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f392806d200 msgr2=0x7f392814b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:22.123 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.118+0000 7f3927fff700 1 --2- 172.21.15.72:0/129286761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f392806d200 0x7f392814b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:22.123 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.118+0000 7f3927fff700 1 -- 172.21.15.72:0/129286761 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39180094a0 con 0x7f3928159990 2023-03-24T23:47:22.123 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.119+0000 7f3927fff700 1 --2- 172.21.15.72:0/129286761 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3928159990 0x7f39280b2fe0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f392000cb50 tx=0x7f392000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:22.123 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.119+0000 7f390ffff700 1 -- 172.21.15.72:0/129286761 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3920003c00 con 0x7f3928159990 2023-03-24T23:47:22.123 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.119+0000 7f390ffff700 1 -- 172.21.15.72:0/129286761 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3920003d60 con 0x7f3928159990 2023-03-24T23:47:22.123 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.119+0000 7f392dc50700 1 -- 172.21.15.72:0/129286761 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f392806c150 con 0x7f3928159990 2023-03-24T23:47:22.124 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.119+0000 7f390ffff700 1 -- 172.21.15.72:0/129286761 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392000aa20 con 0x7f3928159990 2023-03-24T23:47:22.124 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.119+0000 7f392dc50700 1 -- 172.21.15.72:0/129286761 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f392806be70 con 0x7f3928159990 2023-03-24T23:47:22.124 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.119+0000 7f390ffff700 1 -- 172.21.15.72:0/129286761 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f39200050d0 con 0x7f3928159990 2023-03-24T23:47:22.124 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.119+0000 7f390ffff700 1 -- 172.21.15.72:0/129286761 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f3920011d60 con 0x7f3928159990 2023-03-24T23:47:22.124 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.120+0000 7f392dc50700 1 -- 172.21.15.72:0/129286761 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f392815d370 con 0x7f3928159990 2023-03-24T23:47:22.124 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.122+0000 7f390ffff700 1 -- 172.21.15.72:0/129286761 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f392001f060 con 0x7f3928159990 2023-03-24T23:47:22.131 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-03-24T23:47:22.130+0000 7ffbd8287200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-03-24T23:47:22.156 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-24T23:47:22.156+0000 7f18a44d5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-03-24T23:47:22.191 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.191+0000 7f392dc50700 1 -- 172.21.15.72:0/129286761 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f392804e850 con 0x7f3928159990 2023-03-24T23:47:22.192 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.192+0000 7f390ffff700 1 -- 172.21.15.72:0/129286761 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f3920029090 con 0x7f3928159990 2023-03-24T23:47:22.192 INFO:teuthology.misc.health.smithi072.stdout: 2023-03-24T23:47:22.192 INFO:teuthology.misc.health.smithi072.stdout:{"epoch":10,"fsid":"d3c122b7-f0b4-4c96-8f85-86f129e51880","created":"2023-03-24T23:47:18.348997+0000","modified":"2023-03-24T23:47:20.920082+0000","last_up_change":"0.000000","last_in_change":"2023-03-24T23:47:20.920082+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"993abb78-a510-4d76-af32-9015a2b3e5d2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ddfa95f0-f08c-4dce-b961-2e3a9f4bcb3d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"cfcf995d-5a1f-47c1-9691-858f305b1f47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"17c7d98c-55ba-4001-8eac-475f82cc31e1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"c78725a3-3c83-4119-a657-b19b187170e3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"151bcb4f-de54-4e07-be8b-d212f724aec5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e5373fa9-b440-4c05-aa55-1618cc42c18d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"1a67b0d5-8adc-4dd7-825c-6f26334a9a6f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-24T23:47:22.193 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.192+0000 7f392dc50700 1 -- 172.21.15.72:0/129286761 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3928159990 msgr2=0x7f39280b2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:22.193 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.192+0000 7f392dc50700 1 --2- 172.21.15.72:0/129286761 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3928159990 0x7f39280b2fe0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f392000cb50 tx=0x7f392000ce60 comp rx=0 tx=0).stop 2023-03-24T23:47:22.193 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.193+0000 7f392dc50700 1 -- 172.21.15.72:0/129286761 shutdown_connections 2023-03-24T23:47:22.194 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.193+0000 7f392dc50700 1 --2- 172.21.15.72:0/129286761 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3928159990 0x7f39280b2fe0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:22.194 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.193+0000 7f392dc50700 1 --2- 172.21.15.72:0/129286761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f392806d200 0x7f392814b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:22.194 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.193+0000 7f392dc50700 1 --2- 172.21.15.72:0/129286761 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f39280cec60 0x7f39280b5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:22.194 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.193+0000 7f392dc50700 1 -- 172.21.15.72:0/129286761 >> 172.21.15.72:0/129286761 conn(0x7f39280cd590 msgr2=0x7f392815c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:22.194 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.193+0000 7f392dc50700 1 -- 172.21.15.72:0/129286761 shutdown_connections 2023-03-24T23:47:22.194 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:22.193+0000 7f392dc50700 1 -- 172.21.15.72:0/129286761 wait complete. 2023-03-24T23:47:22.203 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-03-24T23:47:22.248 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-03-24T23:47:22.247+0000 7ffbd8287200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-03-24T23:47:22.298 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-24T23:47:22.298+0000 7f18a44d5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-03-24T23:47:22.391 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-03-24T23:47:22.390+0000 7ffbd8287200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-03-24T23:47:22.514 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-24T23:47:22.514+0000 7f18a44d5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-03-24T23:47:22.594 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-24T23:47:22.594+0000 7f18a44d5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-03-24T23:47:22.611 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-03-24T23:47:22.610+0000 7ffbd8287200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-03-24T23:47:22.671 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-24T23:47:22.671+0000 7f18a44d5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-03-24T23:47:22.693 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-03-24T23:47:22.692+0000 7ffbd8287200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-03-24T23:47:22.772 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-03-24T23:47:22.771+0000 7ffbd8287200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-03-24T23:47:23.452 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-24T23:47:23.452+0000 7f18a44d5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-03-24T23:47:23.535 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-24T23:47:23.535+0000 7f18a44d5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-03-24T23:47:23.573 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-03-24T23:47:23.572+0000 7ffbd8287200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-03-24T23:47:23.611 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-24T23:47:23.611+0000 7f18a44d5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-03-24T23:47:23.657 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-03-24T23:47:23.656+0000 7ffbd8287200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-03-24T23:47:23.694 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-24T23:47:23.694+0000 7f18a44d5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-03-24T23:47:23.734 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-03-24T23:47:23.733+0000 7ffbd8287200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-03-24T23:47:23.819 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-03-24T23:47:23.818+0000 7ffbd8287200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-03-24T23:47:23.843 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-24T23:47:23.843+0000 7f18a44d5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-03-24T23:47:23.971 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-03-24T23:47:23.970+0000 7ffbd8287200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-03-24T23:47:24.038 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-24T23:47:24.037+0000 7f18a44d5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-03-24T23:47:24.112 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-24T23:47:24.110+0000 7f18a44d5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-03-24T23:47:24.172 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-03-24T23:47:24.171+0000 7ffbd8287200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-03-24T23:47:24.245 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-03-24T23:47:24.244+0000 7ffbd8287200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-03-24T23:47:24.266 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-24T23:47:24.265+0000 7f18a44d5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-03-24T23:47:24.464 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-03-24T23:47:24.463+0000 7ffbd8287200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-03-24T23:47:25.043 INFO:tasks.ceph.osd.5.smithi146.stderr:2023-03-24T23:47:25.042+0000 7f69fb8b8640 -1 osd.5 0 log_to_monitors true 2023-03-24T23:47:25.070 INFO:tasks.ceph.osd.6.smithi146.stderr:2023-03-24T23:47:25.069+0000 7f3c660ab640 -1 osd.6 0 log_to_monitors true 2023-03-24T23:47:25.075 INFO:tasks.ceph.osd.0.smithi072.stderr:2023-03-24T23:47:25.074+0000 7f78af348640 -1 osd.0 0 log_to_monitors true 2023-03-24T23:47:25.076 INFO:tasks.ceph.osd.4.smithi146.stderr:2023-03-24T23:47:25.075+0000 7f6ccce2c640 -1 osd.4 0 log_to_monitors true 2023-03-24T23:47:25.080 INFO:tasks.ceph.osd.2.smithi072.stderr:2023-03-24T23:47:25.079+0000 7fc191667640 -1 osd.2 0 log_to_monitors true 2023-03-24T23:47:25.094 INFO:tasks.ceph.osd.3.smithi072.stderr:2023-03-24T23:47:25.093+0000 7f9e50761640 -1 osd.3 0 log_to_monitors true 2023-03-24T23:47:25.094 INFO:tasks.ceph.osd.1.smithi072.stderr:2023-03-24T23:47:25.093+0000 7fa92b47b640 -1 osd.1 0 log_to_monitors true 2023-03-24T23:47:25.103 INFO:tasks.ceph.osd.7.smithi146.stderr:2023-03-24T23:47:25.103+0000 7f88973dc640 -1 osd.7 0 log_to_monitors true 2023-03-24T23:47:25.290 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-24T23:47:25.289+0000 7f185dfea700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-03-24T23:47:25.291 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-24T23:47:25.290+0000 7f185dfea700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-03-24T23:47:28.393 INFO:tasks.ceph.osd.7.smithi146.stderr:2023-03-24T23:47:28.392+0000 7f888af82700 -1 osd.7 0 waiting for initial osdmap 2023-03-24T23:47:28.408 INFO:tasks.ceph.osd.7.smithi146.stderr:2023-03-24T23:47:28.407+0000 7f888335d700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-24T23:47:28.425 INFO:tasks.ceph.osd.4.smithi146.stderr:2023-03-24T23:47:28.424+0000 7f6cc09d2700 -1 osd.4 0 waiting for initial osdmap 2023-03-24T23:47:28.442 INFO:tasks.ceph.osd.4.smithi146.stderr:2023-03-24T23:47:28.441+0000 7f6cb8dad700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-24T23:47:28.444 INFO:tasks.ceph.osd.5.smithi146.stderr:2023-03-24T23:47:28.443+0000 7f69f0c61700 -1 osd.5 0 waiting for initial osdmap 2023-03-24T23:47:28.445 INFO:tasks.ceph.osd.6.smithi146.stderr:2023-03-24T23:47:28.445+0000 7f3c59c51700 -1 osd.6 0 waiting for initial osdmap 2023-03-24T23:47:28.462 INFO:tasks.ceph.osd.5.smithi146.stderr:2023-03-24T23:47:28.461+0000 7f69e7839700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-24T23:47:28.462 INFO:tasks.ceph.osd.6.smithi146.stderr:2023-03-24T23:47:28.461+0000 7f3c5202c700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-24T23:47:28.551 INFO:tasks.ceph.osd.2.smithi072.stderr:2023-03-24T23:47:28.550+0000 7fc18520d700 -1 osd.2 0 waiting for initial osdmap 2023-03-24T23:47:28.558 INFO:tasks.ceph.osd.3.smithi072.stderr:2023-03-24T23:47:28.557+0000 7f9e44307700 -1 osd.3 0 waiting for initial osdmap 2023-03-24T23:47:28.560 INFO:tasks.ceph.osd.0.smithi072.stderr:2023-03-24T23:47:28.559+0000 7f78a2eee700 -1 osd.0 0 waiting for initial osdmap 2023-03-24T23:47:28.564 INFO:tasks.ceph.osd.2.smithi072.stderr:2023-03-24T23:47:28.564+0000 7fc17d5e8700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-24T23:47:28.568 INFO:tasks.ceph.osd.3.smithi072.stderr:2023-03-24T23:47:28.568+0000 7f9e3c6e2700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-24T23:47:28.572 INFO:tasks.ceph.osd.0.smithi072.stderr:2023-03-24T23:47:28.572+0000 7f789b2c9700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-24T23:47:28.588 INFO:tasks.ceph.osd.1.smithi072.stderr:2023-03-24T23:47:28.587+0000 7fa91f021700 -1 osd.1 0 waiting for initial osdmap 2023-03-24T23:47:28.602 INFO:tasks.ceph.osd.1.smithi072.stderr:2023-03-24T23:47:28.601+0000 7fa9173fc700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-24T23:47:29.010 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-03-24T23:47:29.124 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.123+0000 7f6f3212b700 1 Processor -- start 2023-03-24T23:47:29.124 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.123+0000 7f6f3212b700 1 -- start start 2023-03-24T23:47:29.127 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.124+0000 7f6f3212b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6f2c0b5f80 0x7f6f2c0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:29.127 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.124+0000 7f6f3212b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6f2c0d0010 0x7f6f2c0d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:29.128 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.124+0000 7f6f3212b700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f2c0d2830 con 0x7f6f2c0b23c0 2023-03-24T23:47:29.128 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.124+0000 7f6f3212b700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f2c0b2790 con 0x7f6f2c0b5f80 2023-03-24T23:47:29.128 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.124+0000 7f6f3212b700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c0d2df0 con 0x7f6f2c0d0010 2023-03-24T23:47:29.128 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.124+0000 7f6f2b7fe700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f6f2c0b23c0 0x7f6f2c0d1710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:52792/0 (socket says 172.21.15.72:52792) 2023-03-24T23:47:29.128 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.124+0000 7f6f2b7fe700 1 -- 172.21.15.72:0/3757959934 learned_addr learned my addr 172.21.15.72:0/3757959934 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-03-24T23:47:29.128 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.124+0000 7f6f2a7fc700 1 -- 172.21.15.72:0/3757959934 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2296073996 0 0) 0x7f6f2c0d2830 con 0x7f6f2c0b23c0 2023-03-24T23:47:29.128 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.124+0000 7f6f2bfff700 1 --2- 172.21.15.72:0/3757959934 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6f2c0d0010 0x7f6f2c0d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:29.129 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.124+0000 7f6f2affd700 1 --2- 172.21.15.72:0/3757959934 >> v2:172.21.15.146:3300/0 conn(0x7f6f2c0b5f80 0x7f6f2c0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:29.129 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.124+0000 7f6f2a7fc700 1 -- 172.21.15.72:0/3757959934 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f140035b0 con 0x7f6f2c0b23c0 2023-03-24T23:47:29.129 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.124+0000 7f6f2a7fc700 1 -- 172.21.15.72:0/3757959934 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2771551077 0 0) 0x7f6f140035b0 con 0x7f6f2c0b23c0 2023-03-24T23:47:29.129 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.124+0000 7f6f2a7fc700 1 -- 172.21.15.72:0/3757959934 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6f2c0d0010 msgr2=0x7f6f2c0d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T23:47:29.129 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.124+0000 7f6f2a7fc700 1 --2- 172.21.15.72:0/3757959934 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6f2c0d0010 0x7f6f2c0d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:29.129 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.124+0000 7f6f2a7fc700 1 -- 172.21.15.72:0/3757959934 >> v2:172.21.15.146:3300/0 conn(0x7f6f2c0b5f80 msgr2=0x7f6f2c0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T23:47:29.129 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.124+0000 7f6f2a7fc700 1 --2- 172.21.15.72:0/3757959934 >> v2:172.21.15.146:3300/0 conn(0x7f6f2c0b5f80 0x7f6f2c0cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:29.130 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.124+0000 7f6f2a7fc700 1 -- 172.21.15.72:0/3757959934 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f2c0b6400 con 0x7f6f2c0b23c0 2023-03-24T23:47:29.130 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.124+0000 7f6f2a7fc700 1 -- 172.21.15.72:0/3757959934 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 293402998 0 0) 0x7f6f1c001670 con 0x7f6f2c0b23c0 2023-03-24T23:47:29.130 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.124+0000 7f6f2a7fc700 1 -- 172.21.15.72:0/3757959934 >> v1:172.21.15.72:6789/0 conn(0x7f6f2c0b23c0 legacy=0x7f6f2c0d1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:29.130 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.124+0000 7f6f2a7fc700 1 --2- 172.21.15.72:0/3757959934 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6f2c0d0010 0x7f6f14003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:29.131 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.124+0000 7f6f2a7fc700 1 --2- 172.21.15.72:0/3757959934 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f14003f80 0x7f6f14004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:29.131 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.125+0000 7f6f2a7fc700 1 --2- 172.21.15.72:0/3757959934 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f14004d90 0x7f6f14005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:29.131 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.125+0000 7f6f2a7fc700 1 -- 172.21.15.72:0/3757959934 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c0b2790 con 0x7f6f14003f80 2023-03-24T23:47:29.131 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.125+0000 7f6f2a7fc700 1 -- 172.21.15.72:0/3757959934 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c0d2df0 con 0x7f6f14004d90 2023-03-24T23:47:29.131 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.125+0000 7f6f2a7fc700 1 -- 172.21.15.72:0/3757959934 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1400ba40 con 0x7f6f2c0d0010 2023-03-24T23:47:29.132 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.125+0000 7f6f2affd700 1 --2- 172.21.15.72:0/3757959934 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f14003f80 0x7f6f14004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:29.132 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.125+0000 7f6f2b7fe700 1 --2- 172.21.15.72:0/3757959934 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6f2c0d0010 0x7f6f14003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:29.132 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.125+0000 7f6f2bfff700 1 --2- 172.21.15.72:0/3757959934 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f14004d90 0x7f6f14005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:29.132 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.125+0000 7f6f2affd700 1 -- 172.21.15.72:0/3757959934 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6f2c0d0010 msgr2=0x7f6f14003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T23:47:29.132 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.125+0000 7f6f2affd700 1 --2- 172.21.15.72:0/3757959934 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6f2c0d0010 0x7f6f14003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:29.132 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.125+0000 7f6f2affd700 1 -- 172.21.15.72:0/3757959934 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f14004d90 msgr2=0x7f6f14005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T23:47:29.132 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.125+0000 7f6f2affd700 1 --2- 172.21.15.72:0/3757959934 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f14004d90 0x7f6f14005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:29.133 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.125+0000 7f6f2affd700 1 -- 172.21.15.72:0/3757959934 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f140035b0 con 0x7f6f14003f80 2023-03-24T23:47:29.133 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.125+0000 7f6f2affd700 1 --2- 172.21.15.72:0/3757959934 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f14003f80 0x7f6f14004850 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6f20009b60 tx=0x7f6f14005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb0e25424f2a99c9 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:29.133 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.125+0000 7f6f2a7fc700 1 -- 172.21.15.72:0/3757959934 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2000e660 con 0x7f6f14003f80 2023-03-24T23:47:29.133 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.125+0000 7f6f2a7fc700 1 -- 172.21.15.72:0/3757959934 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f2000e7c0 con 0x7f6f14003f80 2023-03-24T23:47:29.133 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.125+0000 7f6f2a7fc700 1 -- 172.21.15.72:0/3757959934 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2000eb30 con 0x7f6f14003f80 2023-03-24T23:47:29.133 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.126+0000 7f6f3212b700 1 -- 172.21.15.72:0/3757959934 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f14003f80 msgr2=0x7f6f14004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:29.133 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.126+0000 7f6f3212b700 1 --2- 172.21.15.72:0/3757959934 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f14003f80 0x7f6f14004850 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6f20009b60 tx=0x7f6f14005880 comp rx=0 tx=0).stop 2023-03-24T23:47:29.134 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.126+0000 7f6f2bfff700 1 -- 172.21.15.72:0/3757959934 reap_dead start 2023-03-24T23:47:29.134 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.126+0000 7f6f3212b700 1 -- 172.21.15.72:0/3757959934 shutdown_connections 2023-03-24T23:47:29.134 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.126+0000 7f6f3212b700 1 -- 172.21.15.72:0/3757959934 >> 172.21.15.72:0/3757959934 conn(0x7f6f2c066140 msgr2=0x7f6f2c066540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:29.134 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.126+0000 7f6f3212b700 1 -- 172.21.15.72:0/3757959934 shutdown_connections 2023-03-24T23:47:29.134 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.126+0000 7f6f3212b700 1 -- 172.21.15.72:0/3757959934 wait complete. 2023-03-24T23:47:29.136 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.127+0000 7f6f3212b700 1 Processor -- start 2023-03-24T23:47:29.136 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.128+0000 7f6f3212b700 1 -- start start 2023-03-24T23:47:29.136 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.128+0000 7f6f3212b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f14003f80 0x7f6f2c154100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:29.137 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.128+0000 7f6f3212b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6f2c0c8710 0x7f6f2c0c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:29.137 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.128+0000 7f6f3212b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f2c0c8b80 0x7f6f2c0c7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:29.137 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.128+0000 7f6f3212b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c154c40 con 0x7f6f14003f80 2023-03-24T23:47:29.137 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.128+0000 7f6f3212b700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c0c7670 con 0x7f6f2c0c8b80 2023-03-24T23:47:29.137 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.128+0000 7f6f3212b700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c0c7950 con 0x7f6f2c0c8710 2023-03-24T23:47:29.137 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.128+0000 7f6f2b7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f14003f80 0x7f6f2c154100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:29.138 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.129+0000 7f6f2b7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f14003f80 0x7f6f2c154100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56240/0 (socket says 172.21.15.72:56240) 2023-03-24T23:47:29.138 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.129+0000 7f6f2b7fe700 1 -- 172.21.15.72:0/298019988 learned_addr learned my addr 172.21.15.72:0/298019988 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-03-24T23:47:29.138 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.129+0000 7f6f2affd700 1 --2- 172.21.15.72:0/298019988 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6f2c0c8710 0x7f6f2c0c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:29.138 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.129+0000 7f6f2bfff700 1 --2- 172.21.15.72:0/298019988 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f2c0c8b80 0x7f6f2c0c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:29.138 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.129+0000 7f6f2b7fe700 1 -- 172.21.15.72:0/298019988 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6f2c0c8710 msgr2=0x7f6f2c0c6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:29.138 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.129+0000 7f6f2b7fe700 1 --2- 172.21.15.72:0/298019988 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6f2c0c8710 0x7f6f2c0c6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:29.138 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.129+0000 7f6f2b7fe700 1 -- 172.21.15.72:0/298019988 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f2c0c8b80 msgr2=0x7f6f2c0c7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:29.139 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.129+0000 7f6f2b7fe700 1 --2- 172.21.15.72:0/298019988 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f2c0c8b80 0x7f6f2c0c7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:29.139 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.129+0000 7f6f2b7fe700 1 -- 172.21.15.72:0/298019988 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f2c0cd920 con 0x7f6f14003f80 2023-03-24T23:47:29.139 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.129+0000 7f6f2bfff700 1 --2- 172.21.15.72:0/298019988 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f2c0c8b80 0x7f6f2c0c7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-24T23:47:29.139 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.129+0000 7f6f2b7fe700 1 --2- 172.21.15.72:0/298019988 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f14003f80 0x7f6f2c154100 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c007310 tx=0x7f6f1c007620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:29.139 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.129+0000 7f6f13fff700 1 -- 172.21.15.72:0/298019988 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c011580 con 0x7f6f14003f80 2023-03-24T23:47:29.139 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.129+0000 7f6f13fff700 1 -- 172.21.15.72:0/298019988 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f1c0116e0 con 0x7f6f14003f80 2023-03-24T23:47:29.139 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.129+0000 7f6f3212b700 1 -- 172.21.15.72:0/298019988 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f2c0d0450 con 0x7f6f14003f80 2023-03-24T23:47:29.140 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.129+0000 7f6f13fff700 1 -- 172.21.15.72:0/298019988 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c011a90 con 0x7f6f14003f80 2023-03-24T23:47:29.140 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.129+0000 7f6f3212b700 1 -- 172.21.15.72:0/298019988 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f2c0cdc00 con 0x7f6f14003f80 2023-03-24T23:47:29.140 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.130+0000 7f6f3212b700 1 -- 172.21.15.72:0/298019988 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f2c04e850 con 0x7f6f14003f80 2023-03-24T23:47:29.140 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.130+0000 7f6f13fff700 1 -- 172.21.15.72:0/298019988 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 87553+0+0 (secure 0 0 0) 0x7f6f1c011bf0 con 0x7f6f14003f80 2023-03-24T23:47:29.140 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.131+0000 7f6f13fff700 1 --2- 172.21.15.72:0/298019988 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f6f18069d50 0x7f6f1806c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:29.140 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.131+0000 7f6f2affd700 1 --2- 172.21.15.72:0/298019988 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f6f18069d50 0x7f6f1806c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:29.140 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.131+0000 7f6f13fff700 1 -- 172.21.15.72:0/298019988 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f6f1c086e80 con 0x7f6f14003f80 2023-03-24T23:47:29.141 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.131+0000 7f6f2affd700 1 --2- 172.21.15.72:0/298019988 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f6f18069d50 0x7f6f1806c200 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6f200090b0 tx=0x7f6f20008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:29.142 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.135+0000 7f6f13fff700 1 -- 172.21.15.72:0/298019988 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f6f1c053840 con 0x7f6f14003f80 2023-03-24T23:47:29.323 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.322+0000 7f6f3212b700 1 -- 172.21.15.72:0/298019988 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f2c160170 con 0x7f6f14003f80 2023-03-24T23:47:29.323 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.322+0000 7f6f13fff700 1 -- 172.21.15.72:0/298019988 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f6f1c056e60 con 0x7f6f14003f80 2023-03-24T23:47:29.324 INFO:teuthology.misc.health.smithi072.stdout: 2023-03-24T23:47:29.324 INFO:teuthology.misc.health.smithi072.stdout:{"epoch":12,"fsid":"d3c122b7-f0b4-4c96-8f85-86f129e51880","created":"2023-03-24T23:47:18.348997+0000","modified":"2023-03-24T23:47:26.367307+0000","last_up_change":"0.000000","last_in_change":"2023-03-24T23:47:20.920082+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"993abb78-a510-4d76-af32-9015a2b3e5d2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ddfa95f0-f08c-4dce-b961-2e3a9f4bcb3d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"cfcf995d-5a1f-47c1-9691-858f305b1f47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"17c7d98c-55ba-4001-8eac-475f82cc31e1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"c78725a3-3c83-4119-a657-b19b187170e3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"151bcb4f-de54-4e07-be8b-d212f724aec5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e5373fa9-b440-4c05-aa55-1618cc42c18d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"1a67b0d5-8adc-4dd7-825c-6f26334a9a6f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-24T23:47:29.325 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.324+0000 7f6f3212b700 1 -- 172.21.15.72:0/298019988 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f6f18069d50 msgr2=0x7f6f1806c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:29.326 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.324+0000 7f6f3212b700 1 --2- 172.21.15.72:0/298019988 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f6f18069d50 0x7f6f1806c200 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6f200090b0 tx=0x7f6f20008030 comp rx=0 tx=0).stop 2023-03-24T23:47:29.326 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.324+0000 7f6f3212b700 1 -- 172.21.15.72:0/298019988 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f14003f80 msgr2=0x7f6f2c154100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:29.326 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.324+0000 7f6f3212b700 1 --2- 172.21.15.72:0/298019988 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f14003f80 0x7f6f2c154100 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c007310 tx=0x7f6f1c007620 comp rx=0 tx=0).stop 2023-03-24T23:47:29.326 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.324+0000 7f6f3212b700 1 -- 172.21.15.72:0/298019988 shutdown_connections 2023-03-24T23:47:29.326 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.324+0000 7f6f3212b700 1 --2- 172.21.15.72:0/298019988 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f6f18069d50 0x7f6f1806c200 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:29.326 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.324+0000 7f6f3212b700 1 --2- 172.21.15.72:0/298019988 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f2c0c8b80 0x7f6f2c0c7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:29.326 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.324+0000 7f6f3212b700 1 --2- 172.21.15.72:0/298019988 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f14003f80 0x7f6f2c154100 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:29.327 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.324+0000 7f6f3212b700 1 --2- 172.21.15.72:0/298019988 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6f2c0c8710 0x7f6f2c0c6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:29.327 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.324+0000 7f6f3212b700 1 -- 172.21.15.72:0/298019988 >> 172.21.15.72:0/298019988 conn(0x7f6f2c066140 msgr2=0x7f6f2c15b090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:29.327 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.324+0000 7f6f3212b700 1 -- 172.21.15.72:0/298019988 shutdown_connections 2023-03-24T23:47:29.327 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:29.324+0000 7f6f3212b700 1 -- 172.21.15.72:0/298019988 wait complete. 2023-03-24T23:47:29.337 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-03-24T23:47:36.143 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-03-24T23:47:36.267 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.266+0000 7ff444ef8700 1 Processor -- start 2023-03-24T23:47:36.268 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.266+0000 7ff444ef8700 1 -- start start 2023-03-24T23:47:36.269 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.267+0000 7ff444ef8700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff44015de90 0x7ff4400cf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:36.270 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.267+0000 7ff444ef8700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff4400b6bd0 0x7ff4400cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:36.270 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.267+0000 7ff444ef8700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ff4400b6680 0x7ff4400d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:36.270 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.267+0000 7ff444ef8700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4400cea40 con 0x7ff4400b6680 2023-03-24T23:47:36.270 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.267+0000 7ff444ef8700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4400cf9d0 con 0x7ff4400b6bd0 2023-03-24T23:47:36.270 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.267+0000 7ff444ef8700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff44015edc0 con 0x7ff44015de90 2023-03-24T23:47:36.270 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.268+0000 7ff43ed9d700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ff4400b6680 0x7ff4400d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:36.270 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.268+0000 7ff43ed9d700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ff4400b6680 0x7ff4400d0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37742/0 (socket says 172.21.15.72:37742) 2023-03-24T23:47:36.271 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.268+0000 7ff43ed9d700 1 -- 172.21.15.72:0/581816104 learned_addr learned my addr 172.21.15.72:0/581816104 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-03-24T23:47:36.271 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.268+0000 7ff43e59c700 1 --2- 172.21.15.72:0/581816104 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff44015de90 0x7ff4400cf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:36.271 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.268+0000 7ff43dd9b700 1 --2- 172.21.15.72:0/581816104 >> v2:172.21.15.146:3300/0 conn(0x7ff4400b6bd0 0x7ff4400cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:36.271 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.268+0000 7ff43ed9d700 1 -- 172.21.15.72:0/581816104 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff44015de90 msgr2=0x7ff4400cf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T23:47:36.271 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.268+0000 7ff43ed9d700 1 --2- 172.21.15.72:0/581816104 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff44015de90 0x7ff4400cf490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:36.271 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.268+0000 7ff43ed9d700 1 -- 172.21.15.72:0/581816104 >> v2:172.21.15.146:3300/0 conn(0x7ff4400b6bd0 msgr2=0x7ff4400cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T23:47:36.272 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.268+0000 7ff43ed9d700 1 --2- 172.21.15.72:0/581816104 >> v2:172.21.15.146:3300/0 conn(0x7ff4400b6bd0 0x7ff4400cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:36.272 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.268+0000 7ff43ed9d700 1 -- 172.21.15.72:0/581816104 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44015f0a0 con 0x7ff4400b6680 2023-03-24T23:47:36.272 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.268+0000 7ff43ed9d700 1 --2- 172.21.15.72:0/581816104 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff4400b6680 0x7ff4400d0460 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff4400b3af0 tx=0x7ff43800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca41f98539c48f66 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:36.272 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.268+0000 7ff43d59a700 1 -- 172.21.15.72:0/581816104 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43800e070 con 0x7ff4400b6680 2023-03-24T23:47:36.272 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.268+0000 7ff43d59a700 1 -- 172.21.15.72:0/581816104 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff43800b9c0 con 0x7ff4400b6680 2023-03-24T23:47:36.272 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.268+0000 7ff43d59a700 1 -- 172.21.15.72:0/581816104 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43800bd70 con 0x7ff4400b6680 2023-03-24T23:47:36.272 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.269+0000 7ff444ef8700 1 -- 172.21.15.72:0/581816104 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff4400b6680 msgr2=0x7ff4400d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:36.273 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.269+0000 7ff444ef8700 1 --2- 172.21.15.72:0/581816104 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff4400b6680 0x7ff4400d0460 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff4400b3af0 tx=0x7ff43800b330 comp rx=0 tx=0).stop 2023-03-24T23:47:36.275 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.269+0000 7ff444ef8700 1 -- 172.21.15.72:0/581816104 shutdown_connections 2023-03-24T23:47:36.275 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.269+0000 7ff444ef8700 1 --2- 172.21.15.72:0/581816104 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff4400b6680 0x7ff4400d0460 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:36.275 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.269+0000 7ff444ef8700 1 --2- 172.21.15.72:0/581816104 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff44015de90 0x7ff4400cf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:36.275 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.269+0000 7ff444ef8700 1 --2- 172.21.15.72:0/581816104 >> v2:172.21.15.146:3300/0 conn(0x7ff4400b6bd0 0x7ff4400cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:36.275 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.269+0000 7ff444ef8700 1 -- 172.21.15.72:0/581816104 >> 172.21.15.72:0/581816104 conn(0x7ff44006e360 msgr2=0x7ff44006e730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:36.275 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.269+0000 7ff444ef8700 1 -- 172.21.15.72:0/581816104 shutdown_connections 2023-03-24T23:47:36.276 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.269+0000 7ff444ef8700 1 -- 172.21.15.72:0/581816104 wait complete. 2023-03-24T23:47:36.276 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.270+0000 7ff444ef8700 1 Processor -- start 2023-03-24T23:47:36.276 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.270+0000 7ff444ef8700 1 -- start start 2023-03-24T23:47:36.276 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.271+0000 7ff444ef8700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff4400b6680 0x7ff440159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:36.276 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.271+0000 7ff444ef8700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4400b6bd0 0x7ff4401e62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:36.276 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.271+0000 7ff444ef8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff44015de90 0x7ff4401e6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:36.276 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.271+0000 7ff444ef8700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff440159f00 con 0x7ff44015de90 2023-03-24T23:47:36.276 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.271+0000 7ff444ef8700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44015a1e0 con 0x7ff4400b6bd0 2023-03-24T23:47:36.277 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.271+0000 7ff444ef8700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff44015a4c0 con 0x7ff4400b6680 2023-03-24T23:47:36.277 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.271+0000 7ff43ed9d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff44015de90 0x7ff4401e6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:36.277 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.271+0000 7ff43ed9d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff44015de90 0x7ff4401e6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37748/0 (socket says 172.21.15.72:37748) 2023-03-24T23:47:36.277 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.271+0000 7ff43ed9d700 1 -- 172.21.15.72:0/1320887560 learned_addr learned my addr 172.21.15.72:0/1320887560 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-03-24T23:47:36.277 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.271+0000 7ff43e59c700 1 --2- 172.21.15.72:0/1320887560 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff4400b6680 0x7ff440159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:36.277 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.271+0000 7ff43dd9b700 1 --2- 172.21.15.72:0/1320887560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4400b6bd0 0x7ff4401e62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:36.277 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.271+0000 7ff43ed9d700 1 -- 172.21.15.72:0/1320887560 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff4400b6680 msgr2=0x7ff440159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:36.278 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.271+0000 7ff43ed9d700 1 --2- 172.21.15.72:0/1320887560 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff4400b6680 0x7ff440159700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:36.278 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.271+0000 7ff43ed9d700 1 -- 172.21.15.72:0/1320887560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4400b6bd0 msgr2=0x7ff4401e62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:36.278 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.271+0000 7ff43ed9d700 1 --2- 172.21.15.72:0/1320887560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4400b6bd0 0x7ff4401e62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:36.278 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.272+0000 7ff43ed9d700 1 -- 172.21.15.72:0/1320887560 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff438009b30 con 0x7ff44015de90 2023-03-24T23:47:36.278 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.272+0000 7ff43ed9d700 1 --2- 172.21.15.72:0/1320887560 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff44015de90 0x7ff4401e6820 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff440067940 tx=0x7ff438004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:36.278 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.272+0000 7ff436ffd700 1 -- 172.21.15.72:0/1320887560 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4380099b0 con 0x7ff44015de90 2023-03-24T23:47:36.278 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.272+0000 7ff436ffd700 1 -- 172.21.15.72:0/1320887560 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff438004250 con 0x7ff44015de90 2023-03-24T23:47:36.279 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.272+0000 7ff444ef8700 1 -- 172.21.15.72:0/1320887560 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4400cef70 con 0x7ff44015de90 2023-03-24T23:47:36.279 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.272+0000 7ff436ffd700 1 -- 172.21.15.72:0/1320887560 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff438004600 con 0x7ff44015de90 2023-03-24T23:47:36.279 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.272+0000 7ff444ef8700 1 -- 172.21.15.72:0/1320887560 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4400cec90 con 0x7ff44015de90 2023-03-24T23:47:36.279 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.273+0000 7ff444ef8700 1 -- 172.21.15.72:0/1320887560 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4400700a0 con 0x7ff44015de90 2023-03-24T23:47:36.279 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.273+0000 7ff436ffd700 1 -- 172.21.15.72:0/1320887560 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7ff438004760 con 0x7ff44015de90 2023-03-24T23:47:36.279 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.274+0000 7ff436ffd700 1 --2- 172.21.15.72:0/1320887560 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7ff420069de0 0x7ff42006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:36.279 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.274+0000 7ff436ffd700 1 -- 172.21.15.72:0/1320887560 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff438088ae0 con 0x7ff44015de90 2023-03-24T23:47:36.279 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.274+0000 7ff43e59c700 1 --2- 172.21.15.72:0/1320887560 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7ff420069de0 0x7ff42006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:36.280 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.274+0000 7ff43e59c700 1 --2- 172.21.15.72:0/1320887560 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7ff420069de0 0x7ff42006c290 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff430006b20 tx=0x7ff430008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:36.280 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.278+0000 7ff436ffd700 1 -- 172.21.15.72:0/1320887560 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7ff4380548d0 con 0x7ff44015de90 2023-03-24T23:47:36.466 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.465+0000 7ff444ef8700 1 -- 172.21.15.72:0/1320887560 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff44004e850 con 0x7ff44015de90 2023-03-24T23:47:36.467 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.466+0000 7ff436ffd700 1 -- 172.21.15.72:0/1320887560 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12478 (secure 0 0 0) 0x7ff43801f090 con 0x7ff44015de90 2023-03-24T23:47:36.467 INFO:teuthology.misc.health.smithi072.stdout: 2023-03-24T23:47:36.467 INFO:teuthology.misc.health.smithi072.stdout:{"epoch":16,"fsid":"d3c122b7-f0b4-4c96-8f85-86f129e51880","created":"2023-03-24T23:47:18.348997+0000","modified":"2023-03-24T23:47:32.576486+0000","last_up_change":"2023-03-24T23:47:29.397334+0000","last_in_change":"2023-03-24T23:47:20.920082+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-24T23:47:30.335434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"993abb78-a510-4d76-af32-9015a2b3e5d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":114934},{"type":"v1","addr":"172.21.15.72:6809","nonce":114934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":114934},{"type":"v1","addr":"172.21.15.72:6811","nonce":114934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":114934},{"type":"v1","addr":"172.21.15.72:6815","nonce":114934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":114934},{"type":"v1","addr":"172.21.15.72:6813","nonce":114934}]},"public_addr":"172.21.15.72:6809/114934","cluster_addr":"172.21.15.72:6811/114934","heartbeat_back_addr":"172.21.15.72:6815/114934","heartbeat_front_addr":"172.21.15.72:6813/114934","state":["exists","up"]},{"osd":1,"uuid":"ddfa95f0-f08c-4dce-b961-2e3a9f4bcb3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":114933},{"type":"v1","addr":"172.21.15.72:6801","nonce":114933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":114933},{"type":"v1","addr":"172.21.15.72:6803","nonce":114933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":114933},{"type":"v1","addr":"172.21.15.72:6807","nonce":114933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":114933},{"type":"v1","addr":"172.21.15.72:6805","nonce":114933}]},"public_addr":"172.21.15.72:6801/114933","cluster_addr":"172.21.15.72:6803/114933","heartbeat_back_addr":"172.21.15.72:6807/114933","heartbeat_front_addr":"172.21.15.72:6805/114933","state":["exists","up"]},{"osd":2,"uuid":"cfcf995d-5a1f-47c1-9691-858f305b1f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":114943},{"type":"v1","addr":"172.21.15.72:6818","nonce":114943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":114943},{"type":"v1","addr":"172.21.15.72:6822","nonce":114943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":114943},{"type":"v1","addr":"172.21.15.72:6830","nonce":114943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":114943},{"type":"v1","addr":"172.21.15.72:6826","nonce":114943}]},"public_addr":"172.21.15.72:6818/114943","cluster_addr":"172.21.15.72:6822/114943","heartbeat_back_addr":"172.21.15.72:6830/114943","heartbeat_front_addr":"172.21.15.72:6826/114943","state":["exists","up"]},{"osd":3,"uuid":"17c7d98c-55ba-4001-8eac-475f82cc31e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6817","nonce":114946},{"type":"v1","addr":"172.21.15.72:6819","nonce":114946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6821","nonce":114946},{"type":"v1","addr":"172.21.15.72:6823","nonce":114946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6829","nonce":114946},{"type":"v1","addr":"172.21.15.72:6831","nonce":114946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6825","nonce":114946},{"type":"v1","addr":"172.21.15.72:6827","nonce":114946}]},"public_addr":"172.21.15.72:6819/114946","cluster_addr":"172.21.15.72:6823/114946","heartbeat_back_addr":"172.21.15.72:6831/114946","heartbeat_front_addr":"172.21.15.72:6827/114946","state":["exists","up"]},{"osd":4,"uuid":"c78725a3-3c83-4119-a657-b19b187170e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":115192},{"type":"v1","addr":"172.21.15.146:6817","nonce":115192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":115192},{"type":"v1","addr":"172.21.15.146:6819","nonce":115192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":115192},{"type":"v1","addr":"172.21.15.146:6823","nonce":115192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":115192},{"type":"v1","addr":"172.21.15.146:6821","nonce":115192}]},"public_addr":"172.21.15.146:6817/115192","cluster_addr":"172.21.15.146:6819/115192","heartbeat_back_addr":"172.21.15.146:6823/115192","heartbeat_front_addr":"172.21.15.146:6821/115192","state":["exists","up"]},{"osd":5,"uuid":"151bcb4f-de54-4e07-be8b-d212f724aec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":115190},{"type":"v1","addr":"172.21.15.146:6801","nonce":115190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":115190},{"type":"v1","addr":"172.21.15.146:6803","nonce":115190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":115190},{"type":"v1","addr":"172.21.15.146:6807","nonce":115190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":115190},{"type":"v1","addr":"172.21.15.146:6805","nonce":115190}]},"public_addr":"172.21.15.146:6801/115190","cluster_addr":"172.21.15.146:6803/115190","heartbeat_back_addr":"172.21.15.146:6807/115190","heartbeat_front_addr":"172.21.15.146:6805/115190","state":["exists","up"]},{"osd":6,"uuid":"e5373fa9-b440-4c05-aa55-1618cc42c18d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":115191},{"type":"v1","addr":"172.21.15.146:6809","nonce":115191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":115191},{"type":"v1","addr":"172.21.15.146:6811","nonce":115191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":115191},{"type":"v1","addr":"172.21.15.146:6815","nonce":115191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":115191},{"type":"v1","addr":"172.21.15.146:6813","nonce":115191}]},"public_addr":"172.21.15.146:6809/115191","cluster_addr":"172.21.15.146:6811/115191","heartbeat_back_addr":"172.21.15.146:6815/115191","heartbeat_front_addr":"172.21.15.146:6813/115191","state":["exists","up"]},{"osd":7,"uuid":"1a67b0d5-8adc-4dd7-825c-6f26334a9a6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":115221},{"type":"v1","addr":"172.21.15.146:6825","nonce":115221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":115221},{"type":"v1","addr":"172.21.15.146:6827","nonce":115221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":115221},{"type":"v1","addr":"172.21.15.146:6831","nonce":115221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":115221},{"type":"v1","addr":"172.21.15.146:6829","nonce":115221}]},"public_addr":"172.21.15.146:6825/115221","cluster_addr":"172.21.15.146:6827/115221","heartbeat_back_addr":"172.21.15.146:6831/115221","heartbeat_front_addr":"172.21.15.146:6829/115221","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T23:47:26.063902+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T23:47:26.140546+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T23:47:26.037678+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T23:47:26.144856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T23:47:26.078748+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T23:47:26.005133+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T23:47:26.071184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T23:47:26.089707+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-24T23:47:36.469 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.468+0000 7ff444ef8700 1 -- 172.21.15.72:0/1320887560 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7ff420069de0 msgr2=0x7ff42006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:36.469 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.468+0000 7ff444ef8700 1 --2- 172.21.15.72:0/1320887560 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7ff420069de0 0x7ff42006c290 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff430006b20 tx=0x7ff430008040 comp rx=0 tx=0).stop 2023-03-24T23:47:36.469 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.468+0000 7ff444ef8700 1 -- 172.21.15.72:0/1320887560 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff44015de90 msgr2=0x7ff4401e6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:36.470 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.468+0000 7ff444ef8700 1 --2- 172.21.15.72:0/1320887560 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff44015de90 0x7ff4401e6820 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff440067940 tx=0x7ff438004ad0 comp rx=0 tx=0).stop 2023-03-24T23:47:36.470 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.468+0000 7ff444ef8700 1 -- 172.21.15.72:0/1320887560 shutdown_connections 2023-03-24T23:47:36.470 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.468+0000 7ff444ef8700 1 --2- 172.21.15.72:0/1320887560 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff44015de90 0x7ff4401e6820 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:36.470 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.468+0000 7ff444ef8700 1 --2- 172.21.15.72:0/1320887560 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff4400b6680 0x7ff440159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:36.470 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.468+0000 7ff444ef8700 1 --2- 172.21.15.72:0/1320887560 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7ff420069de0 0x7ff42006c290 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:36.470 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.468+0000 7ff444ef8700 1 --2- 172.21.15.72:0/1320887560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4400b6bd0 0x7ff4401e62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:36.470 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.468+0000 7ff444ef8700 1 -- 172.21.15.72:0/1320887560 >> 172.21.15.72:0/1320887560 conn(0x7ff44006e360 msgr2=0x7ff4400b5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:36.470 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.468+0000 7ff444ef8700 1 -- 172.21.15.72:0/1320887560 shutdown_connections 2023-03-24T23:47:36.471 INFO:teuthology.misc.health.smithi072.stderr:2023-03-24T23:47:36.468+0000 7ff444ef8700 1 -- 172.21.15.72:0/1320887560 wait complete. 2023-03-24T23:47:36.480 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-03-24T23:47:36.480 INFO:tasks.ceph:Creating RBD pool 2023-03-24T23:47:36.480 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph --cluster ceph osd pool create rbd 8 2023-03-24T23:47:36.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.602+0000 7f669e7d1700 1 Processor -- start 2023-03-24T23:47:36.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.602+0000 7f669e7d1700 1 -- start start 2023-03-24T23:47:36.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.602+0000 7f669e7d1700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f66980b23c0 0x7f66980b5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:36.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.602+0000 7f669e7d1700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f66980d1710 0x7f66980cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:36.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.603+0000 7f669e7d1700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66980d2830 con 0x7f66980d0010 2023-03-24T23:47:36.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.603+0000 7f669e7d1700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66980b2790 con 0x7f66980d1710 2023-03-24T23:47:36.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.603+0000 7f669e7d1700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66980d2df0 con 0x7f66980b23c0 2023-03-24T23:47:36.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.603+0000 7f66977fe700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f66980d1710 0x7f66980cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:36.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.603+0000 7f66977fe700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f66980d1710 0x7f66980cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37762/0 (socket says 172.21.15.72:37762) 2023-03-24T23:47:36.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.603+0000 7f66977fe700 1 -- 172.21.15.72:0/959055139 learned_addr learned my addr 172.21.15.72:0/959055139 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-03-24T23:47:36.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.603+0000 7f66977fe700 1 -- 172.21.15.72:0/959055139 >> v2:172.21.15.146:3300/0 conn(0x7f66980b23c0 msgr2=0x7f66980b5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T23:47:36.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.603+0000 7f66977fe700 1 --2- 172.21.15.72:0/959055139 >> v2:172.21.15.146:3300/0 conn(0x7f66980b23c0 0x7f66980b5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:36.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.603+0000 7f66977fe700 1 -- 172.21.15.72:0/959055139 >> v1:172.21.15.146:6789/0 conn(0x7f66980d0010 legacy=0x7f66980d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T23:47:36.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.603+0000 7f66977fe700 1 -- 172.21.15.72:0/959055139 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66980cd920 con 0x7f66980d1710 2023-03-24T23:47:36.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.603+0000 7f66977fe700 1 --2- 172.21.15.72:0/959055139 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66980d1710 0x7f66980cfad0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6698067810 tx=0x7f668c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f0c2771d1b4771b server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:36.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.603+0000 7f6696ffd700 1 -- 172.21.15.72:0/959055139 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668c004230 con 0x7f66980d1710 2023-03-24T23:47:36.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.603+0000 7f6696ffd700 1 -- 172.21.15.72:0/959055139 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f668c004390 con 0x7f66980d1710 2023-03-24T23:47:36.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.603+0000 7f6696ffd700 1 -- 172.21.15.72:0/959055139 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668c0036a0 con 0x7f66980d1710 2023-03-24T23:47:36.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.604+0000 7f669e7d1700 1 -- 172.21.15.72:0/959055139 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66980d1710 msgr2=0x7f66980cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:36.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.604+0000 7f669e7d1700 1 --2- 172.21.15.72:0/959055139 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66980d1710 0x7f66980cfad0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6698067810 tx=0x7f668c009bd0 comp rx=0 tx=0).stop 2023-03-24T23:47:36.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.604+0000 7f669e7d1700 1 -- 172.21.15.72:0/959055139 shutdown_connections 2023-03-24T23:47:36.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.604+0000 7f669e7d1700 1 --2- 172.21.15.72:0/959055139 >> v2:172.21.15.146:3300/0 conn(0x7f66980b23c0 0x7f66980b5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:36.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.604+0000 7f669e7d1700 1 --2- 172.21.15.72:0/959055139 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66980d1710 0x7f66980cfad0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:36.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.604+0000 7f669e7d1700 1 -- 172.21.15.72:0/959055139 >> 172.21.15.72:0/959055139 conn(0x7f6698066140 msgr2=0x7f6698066540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:36.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.604+0000 7f669e7d1700 1 -- 172.21.15.72:0/959055139 shutdown_connections 2023-03-24T23:47:36.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.604+0000 7f669e7d1700 1 -- 172.21.15.72:0/959055139 wait complete. 2023-03-24T23:47:36.610 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.605+0000 7f669e7d1700 1 Processor -- start 2023-03-24T23:47:36.610 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.605+0000 7f669e7d1700 1 -- start start 2023-03-24T23:47:36.610 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.606+0000 7f669e7d1700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66980b23c0 0x7f66981581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:36.610 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.606+0000 7f669e7d1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66980d0010 0x7f66980cc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:36.611 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.606+0000 7f669e7d1700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66980d1710 0x7f669806b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:36.611 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.606+0000 7f669e7d1700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66981f6be0 con 0x7f66980b23c0 2023-03-24T23:47:36.611 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.606+0000 7f669e7d1700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669815d970 con 0x7f66980d0010 2023-03-24T23:47:36.611 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.606+0000 7f669e7d1700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6698158710 con 0x7f66980d1710 2023-03-24T23:47:36.611 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.606+0000 7f6697fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66980b23c0 0x7f66981581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:36.611 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.606+0000 7f6697fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66980b23c0 0x7f66981581d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37764/0 (socket says 172.21.15.72:37764) 2023-03-24T23:47:36.611 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.606+0000 7f6697fff700 1 -- 172.21.15.72:0/2185408480 learned_addr learned my addr 172.21.15.72:0/2185408480 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-03-24T23:47:36.612 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.606+0000 7f669cd6e700 1 --2- 172.21.15.72:0/2185408480 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66980d1710 0x7f669806b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:36.612 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.606+0000 7f6697fff700 1 -- 172.21.15.72:0/2185408480 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66980d1710 msgr2=0x7f669806b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:36.612 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.606+0000 7f6697fff700 1 --2- 172.21.15.72:0/2185408480 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66980d1710 0x7f669806b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:36.612 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.606+0000 7f66977fe700 1 --2- 172.21.15.72:0/2185408480 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66980d0010 0x7f66980cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:36.612 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.606+0000 7f6697fff700 1 -- 172.21.15.72:0/2185408480 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66980d0010 msgr2=0x7f66980cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:36.612 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.606+0000 7f6697fff700 1 --2- 172.21.15.72:0/2185408480 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66980d0010 0x7f66980cc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:36.612 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.606+0000 7f6697fff700 1 -- 172.21.15.72:0/2185408480 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f668c009550 con 0x7f66980b23c0 2023-03-24T23:47:36.612 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.606+0000 7f669cd6e700 1 --2- 172.21.15.72:0/2185408480 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66980d1710 0x7f669806b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-24T23:47:36.613 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.606+0000 7f6697fff700 1 --2- 172.21.15.72:0/2185408480 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66980b23c0 0x7f66981581d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f668800b4d0 tx=0x7f668800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:36.613 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.607+0000 7f6694ff9700 1 -- 172.21.15.72:0/2185408480 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6688011830 con 0x7f66980b23c0 2023-03-24T23:47:36.613 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.607+0000 7f6694ff9700 1 -- 172.21.15.72:0/2185408480 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6688011990 con 0x7f66980b23c0 2023-03-24T23:47:36.613 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.607+0000 7f669e7d1700 1 -- 172.21.15.72:0/2185408480 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66980d2a80 con 0x7f66980b23c0 2023-03-24T23:47:36.613 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.607+0000 7f6694ff9700 1 -- 172.21.15.72:0/2185408480 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6688011d40 con 0x7f66980b23c0 2023-03-24T23:47:36.613 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.607+0000 7f669e7d1700 1 -- 172.21.15.72:0/2185408480 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66980cdc00 con 0x7f66980b23c0 2023-03-24T23:47:36.613 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.607+0000 7f669e7d1700 1 -- 172.21.15.72:0/2185408480 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66980d0410 con 0x7f66980b23c0 2023-03-24T23:47:36.613 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.608+0000 7f6694ff9700 1 -- 172.21.15.72:0/2185408480 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f668801a400 con 0x7f66980b23c0 2023-03-24T23:47:36.614 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.608+0000 7f6694ff9700 1 --2- 172.21.15.72:0/2185408480 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f6684069de0 0x7f668406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:36.614 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.608+0000 7f6694ff9700 1 -- 172.21.15.72:0/2185408480 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6688087b10 con 0x7f66980b23c0 2023-03-24T23:47:36.614 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.608+0000 7f66977fe700 1 --2- 172.21.15.72:0/2185408480 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f6684069de0 0x7f668406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:36.614 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.609+0000 7f66977fe700 1 --2- 172.21.15.72:0/2185408480 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f6684069de0 0x7f668406c290 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f669806bba0 tx=0x7f668c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:36.614 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.612+0000 7f6694ff9700 1 -- 172.21.15.72:0/2185408480 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f6688053820 con 0x7f66980b23c0 2023-03-24T23:47:36.800 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:36.799+0000 7f669e7d1700 1 -- 172.21.15.72:0/2185408480 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f669804e850 con 0x7f66980b23c0 2023-03-24T23:47:37.600 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.599+0000 7f6694ff9700 1 -- 172.21.15.72:0/2185408480 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v17) v1 ==== 109+0+0 (secure 0 0 0) 0x7f66880150a0 con 0x7f66980b23c0 2023-03-24T23:47:37.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.602+0000 7f669e7d1700 1 -- 172.21.15.72:0/2185408480 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f6684069de0 msgr2=0x7f668406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:37.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.602+0000 7f669e7d1700 1 --2- 172.21.15.72:0/2185408480 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f6684069de0 0x7f668406c290 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f669806bba0 tx=0x7f668c008030 comp rx=0 tx=0).stop 2023-03-24T23:47:37.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.602+0000 7f669e7d1700 1 -- 172.21.15.72:0/2185408480 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66980b23c0 msgr2=0x7f66981581d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:37.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.602+0000 7f669e7d1700 1 --2- 172.21.15.72:0/2185408480 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66980b23c0 0x7f66981581d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f668800b4d0 tx=0x7f668800b890 comp rx=0 tx=0).stop 2023-03-24T23:47:37.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.603+0000 7f669e7d1700 1 -- 172.21.15.72:0/2185408480 shutdown_connections 2023-03-24T23:47:37.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.603+0000 7f669e7d1700 1 --2- 172.21.15.72:0/2185408480 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66980d1710 0x7f669806b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:37.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.603+0000 7f669e7d1700 1 --2- 172.21.15.72:0/2185408480 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66980b23c0 0x7f66981581d0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:37.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.603+0000 7f669e7d1700 1 --2- 172.21.15.72:0/2185408480 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f6684069de0 0x7f668406c290 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:37.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.603+0000 7f669e7d1700 1 --2- 172.21.15.72:0/2185408480 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66980d0010 0x7f66980cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:37.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.603+0000 7f669e7d1700 1 -- 172.21.15.72:0/2185408480 >> 172.21.15.72:0/2185408480 conn(0x7f6698066140 msgr2=0x7f66980b3170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:37.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.603+0000 7f669e7d1700 1 -- 172.21.15.72:0/2185408480 shutdown_connections 2023-03-24T23:47:37.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.603+0000 7f669e7d1700 1 -- 172.21.15.72:0/2185408480 wait complete. 2023-03-24T23:47:37.606 INFO:teuthology.orchestra.run.smithi072.stderr:pool 'rbd' created 2023-03-24T23:47:37.616 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-03-24T23:47:37.741 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.740+0000 7ff25b8eb700 1 Processor -- start 2023-03-24T23:47:37.741 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.740+0000 7ff25b8eb700 1 -- start start 2023-03-24T23:47:37.743 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.741+0000 7ff25b8eb700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff254071890 0x7ff2540d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:37.743 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.741+0000 7ff25b8eb700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2540c9f20 0x7ff2540ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:37.743 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.741+0000 7ff25b8eb700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff254071220 con 0x7ff25415cd30 2023-03-24T23:47:37.743 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.741+0000 7ff25b8eb700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff25406be90 con 0x7ff254071890 2023-03-24T23:47:37.743 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.741+0000 7ff25b8eb700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2540b2c20 con 0x7ff2540c9f20 2023-03-24T23:47:37.743 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.741+0000 7ff258e86700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff254071890 0x7ff2540d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:37.744 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.741+0000 7ff259e88700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2540c9f20 0x7ff2540ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:37.744 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.741+0000 7ff258e86700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff254071890 0x7ff2540d00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.72:44230/0 (socket says 172.21.15.72:44230) 2023-03-24T23:47:37.744 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.741+0000 7ff258e86700 1 -- 172.21.15.72:0/1795181238 learned_addr learned my addr 172.21.15.72:0/1795181238 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-03-24T23:47:37.744 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.741+0000 7ff258e86700 1 -- 172.21.15.72:0/1795181238 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2540c9f20 msgr2=0x7ff2540ca3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T23:47:37.744 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.741+0000 7ff258e86700 1 --2- 172.21.15.72:0/1795181238 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2540c9f20 0x7ff2540ca3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:37.744 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.741+0000 7ff258e86700 1 -- 172.21.15.72:0/1795181238 >> v1:172.21.15.146:6789/0 conn(0x7ff25415cd30 legacy=0x7ff2540b2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:37.744 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.741+0000 7ff258e86700 1 -- 172.21.15.72:0/1795181238 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff254071470 con 0x7ff254071890 2023-03-24T23:47:37.744 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.741+0000 7ff259e88700 1 --2- 172.21.15.72:0/1795181238 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2540c9f20 0x7ff2540ca3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-24T23:47:37.745 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.742+0000 7ff258e86700 1 --2- 172.21.15.72:0/1795181238 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff254071890 0x7ff2540d00c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff2440098f0 tx=0x7ff244009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1208df6dc0e633f3 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:37.745 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.742+0000 7ff24bfff700 1 -- 172.21.15.72:0/1795181238 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff244011830 con 0x7ff254071890 2023-03-24T23:47:37.745 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.742+0000 7ff24bfff700 1 -- 172.21.15.72:0/1795181238 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff244011990 con 0x7ff254071890 2023-03-24T23:47:37.745 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.742+0000 7ff24bfff700 1 -- 172.21.15.72:0/1795181238 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff244011d20 con 0x7ff254071890 2023-03-24T23:47:37.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.742+0000 7ff25b8eb700 1 -- 172.21.15.72:0/1795181238 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff254071890 msgr2=0x7ff2540d00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:37.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.742+0000 7ff25b8eb700 1 --2- 172.21.15.72:0/1795181238 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff254071890 0x7ff2540d00c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff2440098f0 tx=0x7ff244009c00 comp rx=0 tx=0).stop 2023-03-24T23:47:37.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.742+0000 7ff25b8eb700 1 -- 172.21.15.72:0/1795181238 shutdown_connections 2023-03-24T23:47:37.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.742+0000 7ff25b8eb700 1 --2- 172.21.15.72:0/1795181238 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2540c9f20 0x7ff2540ca3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:37.748 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.742+0000 7ff25b8eb700 1 --2- 172.21.15.72:0/1795181238 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff254071890 0x7ff2540d00c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:37.748 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.742+0000 7ff25b8eb700 1 -- 172.21.15.72:0/1795181238 >> 172.21.15.72:0/1795181238 conn(0x7ff25406d5a0 msgr2=0x7ff2540cf5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:37.748 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.742+0000 7ff25b8eb700 1 -- 172.21.15.72:0/1795181238 shutdown_connections 2023-03-24T23:47:37.748 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.742+0000 7ff25b8eb700 1 -- 172.21.15.72:0/1795181238 wait complete. 2023-03-24T23:47:37.748 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.743+0000 7ff25b8eb700 1 Processor -- start 2023-03-24T23:47:37.748 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.744+0000 7ff25b8eb700 1 -- start start 2023-03-24T23:47:37.748 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.744+0000 7ff25b8eb700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff254071890 0x7ff2541512c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:37.749 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.744+0000 7ff25b8eb700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff2540c9f20 0x7ff2540c56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:37.749 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.744+0000 7ff25b8eb700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff25415cd30 0x7ff2540c5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:37.749 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.744+0000 7ff25b8eb700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2540c6160 con 0x7ff2540c9f20 2023-03-24T23:47:37.749 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.744+0000 7ff25b8eb700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25415e160 con 0x7ff25415cd30 2023-03-24T23:47:37.749 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.744+0000 7ff25b8eb700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff25415e400 con 0x7ff254071890 2023-03-24T23:47:37.749 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.744+0000 7ff258e86700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff2540c9f20 0x7ff2540c56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:37.749 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.744+0000 7ff258e86700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff2540c9f20 0x7ff2540c56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37766/0 (socket says 172.21.15.72:37766) 2023-03-24T23:47:37.750 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.744+0000 7ff258e86700 1 -- 172.21.15.72:0/914420189 learned_addr learned my addr 172.21.15.72:0/914420189 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-03-24T23:47:37.750 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.745+0000 7ff259e88700 1 --2- 172.21.15.72:0/914420189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff25415cd30 0x7ff2540c5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:37.750 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.745+0000 7ff259687700 1 --2- 172.21.15.72:0/914420189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff254071890 0x7ff2541512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:37.750 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.745+0000 7ff258e86700 1 -- 172.21.15.72:0/914420189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff254071890 msgr2=0x7ff2541512c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:37.750 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.745+0000 7ff258e86700 1 --2- 172.21.15.72:0/914420189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff254071890 0x7ff2541512c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:37.750 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.745+0000 7ff258e86700 1 -- 172.21.15.72:0/914420189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff25415cd30 msgr2=0x7ff2540c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:37.750 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.745+0000 7ff258e86700 1 --2- 172.21.15.72:0/914420189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff25415cd30 0x7ff2540c5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:37.750 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.745+0000 7ff258e86700 1 -- 172.21.15.72:0/914420189 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff24c009a70 con 0x7ff2540c9f20 2023-03-24T23:47:37.751 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.745+0000 7ff259e88700 1 --2- 172.21.15.72:0/914420189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff25415cd30 0x7ff2540c5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-24T23:47:37.751 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.745+0000 7ff258e86700 1 --2- 172.21.15.72:0/914420189 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff2540c9f20 0x7ff2540c56e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff244009c40 tx=0x7ff244004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:37.751 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.745+0000 7ff249ffb700 1 -- 172.21.15.72:0/914420189 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2440036a0 con 0x7ff2540c9f20 2023-03-24T23:47:37.751 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.745+0000 7ff249ffb700 1 -- 172.21.15.72:0/914420189 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff24401e070 con 0x7ff2540c9f20 2023-03-24T23:47:37.751 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.745+0000 7ff25b8eb700 1 -- 172.21.15.72:0/914420189 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff244009550 con 0x7ff2540c9f20 2023-03-24T23:47:37.751 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.745+0000 7ff249ffb700 1 -- 172.21.15.72:0/914420189 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff244003a60 con 0x7ff2540c9f20 2023-03-24T23:47:37.751 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.745+0000 7ff25b8eb700 1 -- 172.21.15.72:0/914420189 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff25406bb20 con 0x7ff2540c9f20 2023-03-24T23:47:37.752 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.746+0000 7ff25b8eb700 1 -- 172.21.15.72:0/914420189 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff25406b840 con 0x7ff2540c9f20 2023-03-24T23:47:37.752 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.747+0000 7ff249ffb700 1 -- 172.21.15.72:0/914420189 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7ff244003bc0 con 0x7ff2540c9f20 2023-03-24T23:47:37.752 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.747+0000 7ff249ffb700 1 --2- 172.21.15.72:0/914420189 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7ff238069de0 0x7ff23806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:37.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.747+0000 7ff249ffb700 1 -- 172.21.15.72:0/914420189 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5199+0+0 (secure 0 0 0) 0x7ff244088c80 con 0x7ff2540c9f20 2023-03-24T23:47:37.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.747+0000 7ff259687700 1 --2- 172.21.15.72:0/914420189 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7ff238069de0 0x7ff23806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:37.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.747+0000 7ff259687700 1 --2- 172.21.15.72:0/914420189 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7ff238069de0 0x7ff23806c290 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff250000f60 tx=0x7ff250009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:37.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.751+0000 7ff249ffb700 1 -- 172.21.15.72:0/914420189 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7ff2440548a0 con 0x7ff2540c9f20 2023-03-24T23:47:37.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:37.940+0000 7ff25b8eb700 1 -- 172.21.15.72:0/914420189 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff25415d100 con 0x7ff2540c9f20 2023-03-24T23:47:38.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.600+0000 7ff249ffb700 1 -- 172.21.15.72:0/914420189 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v18) v1 ==== 173+0+0 (secure 0 0 0) 0x7ff2440200c0 con 0x7ff2540c9f20 2023-03-24T23:47:38.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.604+0000 7ff25b8eb700 1 -- 172.21.15.72:0/914420189 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7ff238069de0 msgr2=0x7ff23806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:38.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.604+0000 7ff25b8eb700 1 --2- 172.21.15.72:0/914420189 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7ff238069de0 0x7ff23806c290 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff250000f60 tx=0x7ff250009040 comp rx=0 tx=0).stop 2023-03-24T23:47:38.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.604+0000 7ff25b8eb700 1 -- 172.21.15.72:0/914420189 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff2540c9f20 msgr2=0x7ff2540c56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:38.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.604+0000 7ff25b8eb700 1 --2- 172.21.15.72:0/914420189 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff2540c9f20 0x7ff2540c56e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff244009c40 tx=0x7ff244004750 comp rx=0 tx=0).stop 2023-03-24T23:47:38.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.605+0000 7ff25b8eb700 1 -- 172.21.15.72:0/914420189 shutdown_connections 2023-03-24T23:47:38.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.605+0000 7ff25b8eb700 1 --2- 172.21.15.72:0/914420189 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7ff238069de0 0x7ff23806c290 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:38.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.605+0000 7ff25b8eb700 1 --2- 172.21.15.72:0/914420189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff25415cd30 0x7ff2540c5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:38.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.605+0000 7ff25b8eb700 1 --2- 172.21.15.72:0/914420189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff254071890 0x7ff2541512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:38.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.605+0000 7ff25b8eb700 1 --2- 172.21.15.72:0/914420189 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff2540c9f20 0x7ff2540c56e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:38.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.605+0000 7ff25b8eb700 1 -- 172.21.15.72:0/914420189 >> 172.21.15.72:0/914420189 conn(0x7ff25406d5a0 msgr2=0x7ff2540ca3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:38.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.605+0000 7ff25b8eb700 1 -- 172.21.15.72:0/914420189 shutdown_connections 2023-03-24T23:47:38.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.605+0000 7ff25b8eb700 1 -- 172.21.15.72:0/914420189 wait complete. 2023-03-24T23:47:38.608 INFO:teuthology.orchestra.run.smithi072.stderr:enabled application 'rbd' on pool 'rbd' 2023-03-24T23:47:38.621 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-03-24T23:47:38.622 INFO:tasks.ceph.mds.a:Restarting daemon 2023-03-24T23:47:38.622 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-03-24T23:47:38.625 INFO:tasks.ceph.mds.a:Started 2023-03-24T23:47:38.626 INFO:tasks.ceph.mds.c:Restarting daemon 2023-03-24T23:47:38.626 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-03-24T23:47:38.629 INFO:tasks.ceph.mds.c:Started 2023-03-24T23:47:38.630 INFO:tasks.ceph.mds.b:Restarting daemon 2023-03-24T23:47:38.630 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-03-24T23:47:38.633 INFO:tasks.ceph.mds.b:Started 2023-03-24T23:47:38.634 INFO:tasks.ceph.mds.d:Restarting daemon 2023-03-24T23:47:38.634 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-03-24T23:47:38.636 INFO:tasks.ceph.mds.d:Started 2023-03-24T23:47:38.636 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-03-24T23:47:38.637 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-24T23:47:38.708 INFO:tasks.ceph.mds.a.smithi072.stdout:starting mds.a at 2023-03-24T23:47:38.710 INFO:tasks.ceph.mds.c.smithi072.stdout:starting mds.c at 2023-03-24T23:47:38.723 INFO:tasks.ceph.mds.d.smithi146.stdout:starting mds.d at 2023-03-24T23:47:38.723 INFO:tasks.ceph.mds.b.smithi146.stdout:starting mds.b at 2023-03-24T23:47:38.724 INFO:tasks.ceph.mds.b.smithi146.stderr:2023-03-24T23:47:38.723+0000 7f81c2eaf700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6833/0: (98) Address already in use 2023-03-24T23:47:38.768 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.767+0000 7f798d8d3700 1 Processor -- start 2023-03-24T23:47:38.769 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.767+0000 7f798d8d3700 1 -- start start 2023-03-24T23:47:38.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.768+0000 7f798d8d3700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f79880d30b0 0x7f798815d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:38.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.768+0000 7f798d8d3700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79880d0cf0 0x7f79880d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:38.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.768+0000 7f798d8d3700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f79880cfc10 0x7f79880d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:38.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.768+0000 7f798d8d3700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f798815ddc0 con 0x7f79880d30b0 2023-03-24T23:47:38.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.768+0000 7f798d8d3700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79880b5980 con 0x7f79880cfc10 2023-03-24T23:47:38.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.768+0000 7f798d8d3700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79880cd770 con 0x7f79880d0cf0 2023-03-24T23:47:38.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.768+0000 7f7986ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f79880d30b0 0x7f798815d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:38.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.768+0000 7f7986ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f79880d30b0 0x7f798815d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37790/0 (socket says 172.21.15.72:37790) 2023-03-24T23:47:38.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.768+0000 7f7986ffd700 1 -- 172.21.15.72:0/352791189 learned_addr learned my addr 172.21.15.72:0/352791189 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-03-24T23:47:38.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.768+0000 7f7986ffd700 1 -- 172.21.15.72:0/352791189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79880d0cf0 msgr2=0x7f79880d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T23:47:38.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.768+0000 7f7986ffd700 1 --2- 172.21.15.72:0/352791189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79880d0cf0 0x7f79880d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:38.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.769+0000 7f7986ffd700 1 -- 172.21.15.72:0/352791189 >> v2:172.21.15.146:3300/0 conn(0x7f79880cfc10 msgr2=0x7f79880d0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T23:47:38.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.768+0000 7f79877fe700 1 --2- 172.21.15.72:0/352791189 >> v2:172.21.15.146:3300/0 conn(0x7f79880cfc10 0x7f79880d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:38.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.769+0000 7f7986ffd700 1 --2- 172.21.15.72:0/352791189 >> v2:172.21.15.146:3300/0 conn(0x7f79880cfc10 0x7f79880d0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:38.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.769+0000 7f7986ffd700 1 -- 172.21.15.72:0/352791189 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79880cda50 con 0x7f79880d30b0 2023-03-24T23:47:38.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.769+0000 7f79877fe700 1 --2- 172.21.15.72:0/352791189 >> v2:172.21.15.146:3300/0 conn(0x7f79880cfc10 0x7f79880d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-24T23:47:38.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.769+0000 7f7986ffd700 1 --2- 172.21.15.72:0/352791189 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f79880d30b0 0x7f798815d880 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f79880b29b0 tx=0x7f798000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e39a717dd2d4e3f server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:38.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.769+0000 7f7985ffb700 1 -- 172.21.15.72:0/352791189 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798000e070 con 0x7f79880d30b0 2023-03-24T23:47:38.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.769+0000 7f7985ffb700 1 -- 172.21.15.72:0/352791189 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f798000b9c0 con 0x7f79880d30b0 2023-03-24T23:47:38.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.769+0000 7f7985ffb700 1 -- 172.21.15.72:0/352791189 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798000bd70 con 0x7f79880d30b0 2023-03-24T23:47:38.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.769+0000 7f798d8d3700 1 -- 172.21.15.72:0/352791189 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f79880d30b0 msgr2=0x7f798815d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:38.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.769+0000 7f798d8d3700 1 --2- 172.21.15.72:0/352791189 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f79880d30b0 0x7f798815d880 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f79880b29b0 tx=0x7f798000b330 comp rx=0 tx=0).stop 2023-03-24T23:47:38.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.769+0000 7f798d8d3700 1 -- 172.21.15.72:0/352791189 shutdown_connections 2023-03-24T23:47:38.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.769+0000 7f798d8d3700 1 --2- 172.21.15.72:0/352791189 >> v2:172.21.15.146:3300/0 conn(0x7f79880cfc10 0x7f79880d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:38.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.769+0000 7f798d8d3700 1 --2- 172.21.15.72:0/352791189 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f79880d30b0 0x7f798815d880 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:38.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.769+0000 7f798d8d3700 1 --2- 172.21.15.72:0/352791189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79880d0cf0 0x7f79880d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:38.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.769+0000 7f798d8d3700 1 -- 172.21.15.72:0/352791189 >> 172.21.15.72:0/352791189 conn(0x7f7988066140 msgr2=0x7f7988066510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:38.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.769+0000 7f798d8d3700 1 -- 172.21.15.72:0/352791189 shutdown_connections 2023-03-24T23:47:38.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.769+0000 7f798d8d3700 1 -- 172.21.15.72:0/352791189 wait complete. 2023-03-24T23:47:38.777 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.771+0000 7f798d8d3700 1 Processor -- start 2023-03-24T23:47:38.777 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.771+0000 7f798d8d3700 1 -- start start 2023-03-24T23:47:38.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.772+0000 7f798d8d3700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f79880cfc10 0x7f79880cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:38.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.772+0000 7f798d8d3700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79880d0cf0 0x7f79881f23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:38.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.772+0000 7f798d8d3700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79880d30b0 0x7f79880cbe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:38.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.772+0000 7f798d8d3700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79880c6c50 con 0x7f79880cfc10 2023-03-24T23:47:38.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.772+0000 7f798d8d3700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79880cc3d0 con 0x7f79880d0cf0 2023-03-24T23:47:38.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.772+0000 7f798d8d3700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79880c6ea0 con 0x7f79880d30b0 2023-03-24T23:47:38.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.772+0000 7f7986ffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f79880cfc10 0x7f79880cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:38.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.772+0000 7f7986ffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f79880cfc10 0x7f79880cc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37792/0 (socket says 172.21.15.72:37792) 2023-03-24T23:47:38.779 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.772+0000 7f7986ffd700 1 -- 172.21.15.72:0/2644553032 learned_addr learned my addr 172.21.15.72:0/2644553032 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-03-24T23:47:38.779 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.772+0000 7f79867fc700 1 --2- 172.21.15.72:0/2644553032 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79880d0cf0 0x7f79881f23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:38.779 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.772+0000 7f79877fe700 1 --2- 172.21.15.72:0/2644553032 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79880d30b0 0x7f79880cbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:38.779 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.772+0000 7f7986ffd700 1 -- 172.21.15.72:0/2644553032 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79880d30b0 msgr2=0x7f79880cbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:38.779 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.772+0000 7f7986ffd700 1 --2- 172.21.15.72:0/2644553032 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79880d30b0 0x7f79880cbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:38.779 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.772+0000 7f7986ffd700 1 -- 172.21.15.72:0/2644553032 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79880d0cf0 msgr2=0x7f79881f23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:38.779 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.772+0000 7f7986ffd700 1 --2- 172.21.15.72:0/2644553032 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79880d0cf0 0x7f79881f23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:38.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.772+0000 7f7986ffd700 1 -- 172.21.15.72:0/2644553032 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7980009b30 con 0x7f79880cfc10 2023-03-24T23:47:38.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.773+0000 7f7986ffd700 1 --2- 172.21.15.72:0/2644553032 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f79880cfc10 0x7f79880cc9b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f798000b370 tx=0x7f7980004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:38.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.773+0000 7f796ffff700 1 -- 172.21.15.72:0/2644553032 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79800099b0 con 0x7f79880cfc10 2023-03-24T23:47:38.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.773+0000 7f796ffff700 1 -- 172.21.15.72:0/2644553032 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79800092e0 con 0x7f79880cfc10 2023-03-24T23:47:38.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.773+0000 7f798d8d3700 1 -- 172.21.15.72:0/2644553032 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79880cf6f0 con 0x7f79880cfc10 2023-03-24T23:47:38.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.773+0000 7f796ffff700 1 -- 172.21.15.72:0/2644553032 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798001b630 con 0x7f79880cfc10 2023-03-24T23:47:38.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.773+0000 7f798d8d3700 1 -- 172.21.15.72:0/2644553032 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79880cdd30 con 0x7f79880cfc10 2023-03-24T23:47:38.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.774+0000 7f798d8d3700 1 -- 172.21.15.72:0/2644553032 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79880cf370 con 0x7f79880cfc10 2023-03-24T23:47:38.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.774+0000 7f796ffff700 1 -- 172.21.15.72:0/2644553032 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f798001b790 con 0x7f79880cfc10 2023-03-24T23:47:38.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.774+0000 7f796ffff700 1 --2- 172.21.15.72:0/2644553032 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f7974069de0 0x7f797406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:38.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.775+0000 7f79867fc700 1 --2- 172.21.15.72:0/2644553032 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f7974069de0 0x7f797406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:38.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.775+0000 7f796ffff700 1 -- 172.21.15.72:0/2644553032 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7980089330 con 0x7f79880cfc10 2023-03-24T23:47:38.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.775+0000 7f79867fc700 1 --2- 172.21.15.72:0/2644553032 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f7974069de0 0x7f797406c290 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f797c006fd0 tx=0x7f797c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:38.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.779+0000 7f796ffff700 1 -- 172.21.15.72:0/2644553032 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f7980054fc0 con 0x7f79880cfc10 2023-03-24T23:47:38.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.964+0000 7f798d8d3700 1 -- 172.21.15.72:0/2644553032 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f798815d270 con 0x7f79880cfc10 2023-03-24T23:47:38.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.965+0000 7f796ffff700 1 -- 172.21.15.72:0/2644553032 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+14277 (secure 0 0 0) 0x7f79800585e0 con 0x7f79880cfc10 2023-03-24T23:47:38.967 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:47:38.967 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":18,"fsid":"d3c122b7-f0b4-4c96-8f85-86f129e51880","created":"2023-03-24T23:47:18.348997+0000","modified":"2023-03-24T23:47:38.597785+0000","last_up_change":"2023-03-24T23:47:29.397334+0000","last_in_change":"2023-03-24T23:47:20.920082+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-24T23:47:30.335434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-24T23:47:36.817567+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"993abb78-a510-4d76-af32-9015a2b3e5d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":114934},{"type":"v1","addr":"172.21.15.72:6809","nonce":114934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":114934},{"type":"v1","addr":"172.21.15.72:6811","nonce":114934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":114934},{"type":"v1","addr":"172.21.15.72:6815","nonce":114934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":114934},{"type":"v1","addr":"172.21.15.72:6813","nonce":114934}]},"public_addr":"172.21.15.72:6809/114934","cluster_addr":"172.21.15.72:6811/114934","heartbeat_back_addr":"172.21.15.72:6815/114934","heartbeat_front_addr":"172.21.15.72:6813/114934","state":["exists","up"]},{"osd":1,"uuid":"ddfa95f0-f08c-4dce-b961-2e3a9f4bcb3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":114933},{"type":"v1","addr":"172.21.15.72:6801","nonce":114933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":114933},{"type":"v1","addr":"172.21.15.72:6803","nonce":114933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":114933},{"type":"v1","addr":"172.21.15.72:6807","nonce":114933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":114933},{"type":"v1","addr":"172.21.15.72:6805","nonce":114933}]},"public_addr":"172.21.15.72:6801/114933","cluster_addr":"172.21.15.72:6803/114933","heartbeat_back_addr":"172.21.15.72:6807/114933","heartbeat_front_addr":"172.21.15.72:6805/114933","state":["exists","up"]},{"osd":2,"uuid":"cfcf995d-5a1f-47c1-9691-858f305b1f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":114943},{"type":"v1","addr":"172.21.15.72:6818","nonce":114943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":114943},{"type":"v1","addr":"172.21.15.72:6822","nonce":114943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":114943},{"type":"v1","addr":"172.21.15.72:6830","nonce":114943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":114943},{"type":"v1","addr":"172.21.15.72:6826","nonce":114943}]},"public_addr":"172.21.15.72:6818/114943","cluster_addr":"172.21.15.72:6822/114943","heartbeat_back_addr":"172.21.15.72:6830/114943","heartbeat_front_addr":"172.21.15.72:6826/114943","state":["exists","up"]},{"osd":3,"uuid":"17c7d98c-55ba-4001-8eac-475f82cc31e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6817","nonce":114946},{"type":"v1","addr":"172.21.15.72:6819","nonce":114946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6821","nonce":114946},{"type":"v1","addr":"172.21.15.72:6823","nonce":114946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6829","nonce":114946},{"type":"v1","addr":"172.21.15.72:6831","nonce":114946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6825","nonce":114946},{"type":"v1","addr":"172.21.15.72:6827","nonce":114946}]},"public_addr":"172.21.15.72:6819/114946","cluster_addr":"172.21.15.72:6823/114946","heartbeat_back_addr":"172.21.15.72:6831/114946","heartbeat_front_addr":"172.21.15.72:6827/114946","state":["exists","up"]},{"osd":4,"uuid":"c78725a3-3c83-4119-a657-b19b187170e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":115192},{"type":"v1","addr":"172.21.15.146:6817","nonce":115192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":115192},{"type":"v1","addr":"172.21.15.146:6819","nonce":115192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":115192},{"type":"v1","addr":"172.21.15.146:6823","nonce":115192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":115192},{"type":"v1","addr":"172.21.15.146:6821","nonce":115192}]},"public_addr":"172.21.15.146:6817/115192","cluster_addr":"172.21.15.146:6819/115192","heartbeat_back_addr":"172.21.15.146:6823/115192","heartbeat_front_addr":"172.21.15.146:6821/115192","state":["exists","up"]},{"osd":5,"uuid":"151bcb4f-de54-4e07-be8b-d212f724aec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":115190},{"type":"v1","addr":"172.21.15.146:6801","nonce":115190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":115190},{"type":"v1","addr":"172.21.15.146:6803","nonce":115190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":115190},{"type":"v1","addr":"172.21.15.146:6807","nonce":115190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":115190},{"type":"v1","addr":"172.21.15.146:6805","nonce":115190}]},"public_addr":"172.21.15.146:6801/115190","cluster_addr":"172.21.15.146:6803/115190","heartbeat_back_addr":"172.21.15.146:6807/115190","heartbeat_front_addr":"172.21.15.146:6805/115190","state":["exists","up"]},{"osd":6,"uuid":"e5373fa9-b440-4c05-aa55-1618cc42c18d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":115191},{"type":"v1","addr":"172.21.15.146:6809","nonce":115191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":115191},{"type":"v1","addr":"172.21.15.146:6811","nonce":115191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":115191},{"type":"v1","addr":"172.21.15.146:6815","nonce":115191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":115191},{"type":"v1","addr":"172.21.15.146:6813","nonce":115191}]},"public_addr":"172.21.15.146:6809/115191","cluster_addr":"172.21.15.146:6811/115191","heartbeat_back_addr":"172.21.15.146:6815/115191","heartbeat_front_addr":"172.21.15.146:6813/115191","state":["exists","up"]},{"osd":7,"uuid":"1a67b0d5-8adc-4dd7-825c-6f26334a9a6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":115221},{"type":"v1","addr":"172.21.15.146:6825","nonce":115221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":115221},{"type":"v1","addr":"172.21.15.146:6827","nonce":115221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":115221},{"type":"v1","addr":"172.21.15.146:6831","nonce":115221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":115221},{"type":"v1","addr":"172.21.15.146:6829","nonce":115221}]},"public_addr":"172.21.15.146:6825/115221","cluster_addr":"172.21.15.146:6827/115221","heartbeat_back_addr":"172.21.15.146:6831/115221","heartbeat_front_addr":"172.21.15.146:6829/115221","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T23:47:26.063902+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T23:47:26.140546+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T23:47:26.037678+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T23:47:26.144856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T23:47:26.078748+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T23:47:26.005133+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T23:47:26.071184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T23:47:26.089707+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-24T23:47:38.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.967+0000 7f798d8d3700 1 -- 172.21.15.72:0/2644553032 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f7974069de0 msgr2=0x7f797406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:38.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.967+0000 7f798d8d3700 1 --2- 172.21.15.72:0/2644553032 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f7974069de0 0x7f797406c290 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f797c006fd0 tx=0x7f797c008040 comp rx=0 tx=0).stop 2023-03-24T23:47:38.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.967+0000 7f798d8d3700 1 -- 172.21.15.72:0/2644553032 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f79880cfc10 msgr2=0x7f79880cc9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:38.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.967+0000 7f798d8d3700 1 --2- 172.21.15.72:0/2644553032 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f79880cfc10 0x7f79880cc9b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f798000b370 tx=0x7f7980004ad0 comp rx=0 tx=0).stop 2023-03-24T23:47:38.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.967+0000 7f798d8d3700 1 -- 172.21.15.72:0/2644553032 shutdown_connections 2023-03-24T23:47:38.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.967+0000 7f798d8d3700 1 --2- 172.21.15.72:0/2644553032 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79880d30b0 0x7f79880cbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:38.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.967+0000 7f798d8d3700 1 --2- 172.21.15.72:0/2644553032 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f79880cfc10 0x7f79880cc9b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:38.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.967+0000 7f798d8d3700 1 --2- 172.21.15.72:0/2644553032 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f7974069de0 0x7f797406c290 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:38.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.967+0000 7f798d8d3700 1 --2- 172.21.15.72:0/2644553032 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79880d0cf0 0x7f79881f23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:38.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.967+0000 7f798d8d3700 1 -- 172.21.15.72:0/2644553032 >> 172.21.15.72:0/2644553032 conn(0x7f7988066140 msgr2=0x7f79880b31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:38.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.967+0000 7f798d8d3700 1 -- 172.21.15.72:0/2644553032 shutdown_connections 2023-03-24T23:47:38.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:38.967+0000 7f798d8d3700 1 -- 172.21.15.72:0/2644553032 wait complete. 2023-03-24T23:47:38.980 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-03-24T23:47:30.335434+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-03-24T23:47:36.817567+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-03-24T23:47:38.981 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-24T23:47:39.105 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.104+0000 7f103c8f2700 1 Processor -- start 2023-03-24T23:47:39.106 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.104+0000 7f103c8f2700 1 -- start start 2023-03-24T23:47:39.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.105+0000 7f103c8f2700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f10380b5f80 0x7f10380cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:39.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.105+0000 7f103c8f2700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10380d2830 con 0x7f10380b23c0 2023-03-24T23:47:39.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.105+0000 7f103c8f2700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10380d2df0 con 0x7f10380d0010 2023-03-24T23:47:39.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.105+0000 7f103c8f2700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10380b2790 con 0x7f10380b5f80 2023-03-24T23:47:39.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.105+0000 7f103659c700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f10380b23c0 0x7f10380d1710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:51088/0 (socket says 172.21.15.72:51088) 2023-03-24T23:47:39.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.105+0000 7f103659c700 1 -- 172.21.15.72:0/1646592130 learned_addr learned my addr 172.21.15.72:0/1646592130 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-03-24T23:47:39.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.105+0000 7f102dd9b700 1 --2- 172.21.15.72:0/1646592130 >> v2:172.21.15.146:3300/0 conn(0x7f10380b5f80 0x7f10380cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:39.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.106+0000 7f1035d9b700 1 -- 172.21.15.72:0/1646592130 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2123519165 0 0) 0x7f10380d2830 con 0x7f10380b23c0 2023-03-24T23:47:39.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.106+0000 7f1035d9b700 1 -- 172.21.15.72:0/1646592130 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10240035b0 con 0x7f10380b23c0 2023-03-24T23:47:39.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.106+0000 7f1035d9b700 1 -- 172.21.15.72:0/1646592130 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1406436076 0 0) 0x7f10380d2df0 con 0x7f10380d0010 2023-03-24T23:47:39.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.106+0000 7f1035d9b700 1 -- 172.21.15.72:0/1646592130 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10380d2830 con 0x7f10380d0010 2023-03-24T23:47:39.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.106+0000 7f1035d9b700 1 -- 172.21.15.72:0/1646592130 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2146340221 0 0) 0x7f10240035b0 con 0x7f10380b23c0 2023-03-24T23:47:39.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.106+0000 7f1035d9b700 1 -- 172.21.15.72:0/1646592130 >> v2:172.21.15.146:3300/0 conn(0x7f10380b5f80 msgr2=0x7f10380cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T23:47:39.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.106+0000 7f1035d9b700 1 --2- 172.21.15.72:0/1646592130 >> v2:172.21.15.146:3300/0 conn(0x7f10380b5f80 0x7f10380cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:39.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.106+0000 7f1035d9b700 1 -- 172.21.15.72:0/1646592130 >> v1:172.21.15.146:6789/0 conn(0x7f10380d0010 legacy=0x7f10380d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:39.111 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.106+0000 7f1035d9b700 1 -- 172.21.15.72:0/1646592130 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10380b6400 con 0x7f10380b23c0 2023-03-24T23:47:39.111 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.106+0000 7f1035d9b700 1 -- 172.21.15.72:0/1646592130 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 293402998 0 0) 0x7f1028001670 con 0x7f10380b23c0 2023-03-24T23:47:39.111 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.106+0000 7f1035d9b700 1 -- 172.21.15.72:0/1646592130 >> v1:172.21.15.72:6789/0 conn(0x7f10380b23c0 legacy=0x7f10380d1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:39.111 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.106+0000 7f1035d9b700 1 --2- 172.21.15.72:0/1646592130 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1024003be0 0x7f1024003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:39.111 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.106+0000 7f1035d9b700 1 --2- 172.21.15.72:0/1646592130 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1024004850 0x7f1024004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:39.111 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.106+0000 7f1035d9b700 1 --2- 172.21.15.72:0/1646592130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1024005250 0x7f102400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:39.111 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.106+0000 7f1035d9b700 1 -- 172.21.15.72:0/1646592130 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10380b2790 con 0x7f1024004850 2023-03-24T23:47:39.111 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.106+0000 7f1035d9b700 1 -- 172.21.15.72:0/1646592130 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102400c160 con 0x7f1024005250 2023-03-24T23:47:39.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.106+0000 7f1035d9b700 1 -- 172.21.15.72:0/1646592130 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f102400c320 con 0x7f1024003be0 2023-03-24T23:47:39.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.106+0000 7f102dd9b700 1 --2- 172.21.15.72:0/1646592130 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1024004850 0x7f1024004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:39.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.106+0000 7f103659c700 1 --2- 172.21.15.72:0/1646592130 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1024003be0 0x7f1024003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:39.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.106+0000 7f1036d9d700 1 --2- 172.21.15.72:0/1646592130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1024005250 0x7f102400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:39.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.106+0000 7f102dd9b700 1 -- 172.21.15.72:0/1646592130 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1024003be0 msgr2=0x7f1024003fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T23:47:39.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.106+0000 7f102dd9b700 1 --2- 172.21.15.72:0/1646592130 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1024003be0 0x7f1024003fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:39.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.106+0000 7f102dd9b700 1 -- 172.21.15.72:0/1646592130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1024005250 msgr2=0x7f102400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T23:47:39.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.106+0000 7f102dd9b700 1 --2- 172.21.15.72:0/1646592130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1024005250 0x7f102400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:39.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.106+0000 7f1036d9d700 1 -- 172.21.15.72:0/1646592130 reap_dead start 2023-03-24T23:47:39.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.106+0000 7f102dd9b700 1 -- 172.21.15.72:0/1646592130 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10240035b0 con 0x7f1024004850 2023-03-24T23:47:39.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.106+0000 7f102dd9b700 1 --2- 172.21.15.72:0/1646592130 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1024004850 0x7f1024004cc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1020009b30 tx=0x7f1024005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46bd0ad7af32b7f3 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:39.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.107+0000 7f1035d9b700 1 -- 172.21.15.72:0/1646592130 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102000e6c0 con 0x7f1024004850 2023-03-24T23:47:39.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.107+0000 7f1035d9b700 1 -- 172.21.15.72:0/1646592130 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f102000e820 con 0x7f1024004850 2023-03-24T23:47:39.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.107+0000 7f1035d9b700 1 -- 172.21.15.72:0/1646592130 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102000eb90 con 0x7f1024004850 2023-03-24T23:47:39.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.107+0000 7f103c8f2700 1 -- 172.21.15.72:0/1646592130 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1024004850 msgr2=0x7f1024004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:39.114 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.107+0000 7f103c8f2700 1 --2- 172.21.15.72:0/1646592130 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1024004850 0x7f1024004cc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1020009b30 tx=0x7f1024005880 comp rx=0 tx=0).stop 2023-03-24T23:47:39.114 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.107+0000 7f103c8f2700 1 -- 172.21.15.72:0/1646592130 shutdown_connections 2023-03-24T23:47:39.114 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.107+0000 7f103c8f2700 1 --2- 172.21.15.72:0/1646592130 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1024004850 0x7f1024004cc0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:39.114 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.107+0000 7f103c8f2700 1 -- 172.21.15.72:0/1646592130 >> 172.21.15.72:0/1646592130 conn(0x7f1038066140 msgr2=0x7f1038066540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:39.114 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.107+0000 7f103c8f2700 1 -- 172.21.15.72:0/1646592130 shutdown_connections 2023-03-24T23:47:39.114 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.107+0000 7f103c8f2700 1 -- 172.21.15.72:0/1646592130 wait complete. 2023-03-24T23:47:39.116 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.109+0000 7f103c8f2700 1 Processor -- start 2023-03-24T23:47:39.116 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.109+0000 7f103c8f2700 1 -- start start 2023-03-24T23:47:39.116 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.109+0000 7f103c8f2700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1024004850 0x7f103815ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:39.117 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.109+0000 7f103c8f2700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1038149510 0x7f10380c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:39.117 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.109+0000 7f103c8f2700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f10380c9230 0x7f10380ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:39.117 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.109+0000 7f103c8f2700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10380c96d0 con 0x7f1024004850 2023-03-24T23:47:39.117 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.109+0000 7f103c8f2700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10380cb200 con 0x7f1038149510 2023-03-24T23:47:39.117 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.110+0000 7f103c8f2700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10380cb4e0 con 0x7f10380c9230 2023-03-24T23:47:39.117 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.110+0000 7f103659c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1024004850 0x7f103815ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:39.117 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.110+0000 7f103659c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1024004850 0x7f103815ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37810/0 (socket says 172.21.15.72:37810) 2023-03-24T23:47:39.117 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.110+0000 7f103659c700 1 -- 172.21.15.72:0/3485835796 learned_addr learned my addr 172.21.15.72:0/3485835796 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-03-24T23:47:39.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.110+0000 7f102dd9b700 1 --2- 172.21.15.72:0/3485835796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1038149510 0x7f10380c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:39.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.110+0000 7f1036d9d700 1 --2- 172.21.15.72:0/3485835796 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f10380c9230 0x7f10380ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:39.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.110+0000 7f103659c700 1 -- 172.21.15.72:0/3485835796 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f10380c9230 msgr2=0x7f10380ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:39.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.110+0000 7f103659c700 1 --2- 172.21.15.72:0/3485835796 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f10380c9230 0x7f10380ca580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:39.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.110+0000 7f103659c700 1 -- 172.21.15.72:0/3485835796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1038149510 msgr2=0x7f10380c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:39.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.110+0000 7f103659c700 1 --2- 172.21.15.72:0/3485835796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1038149510 0x7f10380c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:39.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.110+0000 7f103659c700 1 -- 172.21.15.72:0/3485835796 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10380cdc00 con 0x7f1024004850 2023-03-24T23:47:39.119 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.110+0000 7f103659c700 1 --2- 172.21.15.72:0/3485835796 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1024004850 0x7f103815ce80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1028004f80 tx=0x7f1028007340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:39.119 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.110+0000 7f102f7fe700 1 -- 172.21.15.72:0/3485835796 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1028012560 con 0x7f1024004850 2023-03-24T23:47:39.119 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.110+0000 7f102f7fe700 1 -- 172.21.15.72:0/3485835796 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10280126c0 con 0x7f1024004850 2023-03-24T23:47:39.119 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.110+0000 7f102f7fe700 1 -- 172.21.15.72:0/3485835796 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1028012a70 con 0x7f1024004850 2023-03-24T23:47:39.119 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.110+0000 7f103c8f2700 1 -- 172.21.15.72:0/3485835796 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10380cd920 con 0x7f1024004850 2023-03-24T23:47:39.119 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.110+0000 7f103c8f2700 1 -- 172.21.15.72:0/3485835796 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10380d2a80 con 0x7f1024004850 2023-03-24T23:47:39.119 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.111+0000 7f103c8f2700 1 -- 172.21.15.72:0/3485835796 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10380cc0f0 con 0x7f1024004850 2023-03-24T23:47:39.119 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.112+0000 7f102f7fe700 1 -- 172.21.15.72:0/3485835796 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f1028012bd0 con 0x7f1024004850 2023-03-24T23:47:39.120 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.112+0000 7f102f7fe700 1 --2- 172.21.15.72:0/3485835796 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f1018069de0 0x7f101806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:39.120 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.112+0000 7f102f7fe700 1 -- 172.21.15.72:0/3485835796 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f1028087b80 con 0x7f1024004850 2023-03-24T23:47:39.120 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.112+0000 7f102dd9b700 1 --2- 172.21.15.72:0/3485835796 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f1018069de0 0x7f101806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:39.120 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.112+0000 7f102dd9b700 1 --2- 172.21.15.72:0/3485835796 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f1018069de0 0x7f101806c290 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f10200090b0 tx=0x7f102001f040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:39.120 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.116+0000 7f102f7fe700 1 -- 172.21.15.72:0/3485835796 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f1028053780 con 0x7f1024004850 2023-03-24T23:47:39.298 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.297+0000 7f103c8f2700 1 -- 172.21.15.72:0/3485835796 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f103804e850 con 0x7f1024004850 2023-03-24T23:47:39.299 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.297+0000 7f102f7fe700 1 -- 172.21.15.72:0/3485835796 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1028056da0 con 0x7f1024004850 2023-03-24T23:47:39.299 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2023-03-24T23:47:39.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.299+0000 7f103c8f2700 1 -- 172.21.15.72:0/3485835796 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f1018069de0 msgr2=0x7f101806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:39.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.299+0000 7f103c8f2700 1 --2- 172.21.15.72:0/3485835796 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f1018069de0 0x7f101806c290 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f10200090b0 tx=0x7f102001f040 comp rx=0 tx=0).stop 2023-03-24T23:47:39.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.299+0000 7f103c8f2700 1 -- 172.21.15.72:0/3485835796 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1024004850 msgr2=0x7f103815ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:39.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.299+0000 7f103c8f2700 1 --2- 172.21.15.72:0/3485835796 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1024004850 0x7f103815ce80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1028004f80 tx=0x7f1028007340 comp rx=0 tx=0).stop 2023-03-24T23:47:39.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.300+0000 7f103c8f2700 1 -- 172.21.15.72:0/3485835796 shutdown_connections 2023-03-24T23:47:39.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.300+0000 7f103c8f2700 1 --2- 172.21.15.72:0/3485835796 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f10380c9230 0x7f10380ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:39.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.300+0000 7f103c8f2700 1 --2- 172.21.15.72:0/3485835796 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1024004850 0x7f103815ce80 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:39.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.300+0000 7f103c8f2700 1 --2- 172.21.15.72:0/3485835796 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f1018069de0 0x7f101806c290 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:39.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.300+0000 7f103c8f2700 1 --2- 172.21.15.72:0/3485835796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1038149510 0x7f10380c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:39.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.300+0000 7f103c8f2700 1 -- 172.21.15.72:0/3485835796 >> 172.21.15.72:0/3485835796 conn(0x7f1038066140 msgr2=0x7f1038066510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:39.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.300+0000 7f103c8f2700 1 -- 172.21.15.72:0/3485835796 shutdown_connections 2023-03-24T23:47:39.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.300+0000 7f103c8f2700 1 -- 172.21.15.72:0/3485835796 wait complete. 2023-03-24T23:47:39.313 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-24T23:47:39.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.435+0000 7f399d462700 1 Processor -- start 2023-03-24T23:47:39.437 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.435+0000 7f399d462700 1 -- start start 2023-03-24T23:47:39.438 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.436+0000 7f399d462700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f399806d4a0 0x7f399806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:39.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.436+0000 7f399d462700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f39980cad50 0x7f39980cb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:39.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.436+0000 7f399d462700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f399806d010 con 0x7f399806dae0 2023-03-24T23:47:39.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.436+0000 7f399d462700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39980aef70 con 0x7f39980cad50 2023-03-24T23:47:39.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.436+0000 7f399d462700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39980d0160 con 0x7f399806d4a0 2023-03-24T23:47:39.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.436+0000 7f39977fe700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f39980cad50 0x7f39980cb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:39.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.436+0000 7f39977fe700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f39980cad50 0x7f39980cb190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37824/0 (socket says 172.21.15.72:37824) 2023-03-24T23:47:39.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.436+0000 7f39977fe700 1 -- 172.21.15.72:0/4278295186 learned_addr learned my addr 172.21.15.72:0/4278295186 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-03-24T23:47:39.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.437+0000 7f39977fe700 1 -- 172.21.15.72:0/4278295186 >> v2:172.21.15.146:3300/0 conn(0x7f399806d4a0 msgr2=0x7f399806b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T23:47:39.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.437+0000 7f39967fc700 1 --2- 172.21.15.72:0/4278295186 >> v2:172.21.15.146:3300/0 conn(0x7f399806d4a0 0x7f399806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:39.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.437+0000 7f39977fe700 1 --2- 172.21.15.72:0/4278295186 >> v2:172.21.15.146:3300/0 conn(0x7f399806d4a0 0x7f399806b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:39.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.437+0000 7f39977fe700 1 -- 172.21.15.72:0/4278295186 >> v1:172.21.15.146:6789/0 conn(0x7f399806dae0 legacy=0x7f399806deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T23:47:39.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.437+0000 7f39977fe700 1 -- 172.21.15.72:0/4278295186 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39980d0440 con 0x7f39980cad50 2023-03-24T23:47:39.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.437+0000 7f39977fe700 1 --2- 172.21.15.72:0/4278295186 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f39980cad50 0x7f39980cb190 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f399806e940 tx=0x7f399000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9814e0d2a5175d89 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:39.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.437+0000 7f3995ffb700 1 -- 172.21.15.72:0/4278295186 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399000e070 con 0x7f39980cad50 2023-03-24T23:47:39.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.437+0000 7f3995ffb700 1 -- 172.21.15.72:0/4278295186 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3990003a20 con 0x7f39980cad50 2023-03-24T23:47:39.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.437+0000 7f3995ffb700 1 -- 172.21.15.72:0/4278295186 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3990003dd0 con 0x7f39980cad50 2023-03-24T23:47:39.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.437+0000 7f399d462700 1 -- 172.21.15.72:0/4278295186 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f39980cad50 msgr2=0x7f39980cb190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:39.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.437+0000 7f399d462700 1 --2- 172.21.15.72:0/4278295186 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f39980cad50 0x7f39980cb190 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f399806e940 tx=0x7f399000b330 comp rx=0 tx=0).stop 2023-03-24T23:47:39.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.437+0000 7f399d462700 1 -- 172.21.15.72:0/4278295186 shutdown_connections 2023-03-24T23:47:39.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.438+0000 7f399d462700 1 --2- 172.21.15.72:0/4278295186 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f39980cad50 0x7f39980cb190 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:39.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.438+0000 7f399d462700 1 --2- 172.21.15.72:0/4278295186 >> v2:172.21.15.146:3300/0 conn(0x7f399806d4a0 0x7f399806b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:39.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.438+0000 7f399d462700 1 -- 172.21.15.72:0/4278295186 >> 172.21.15.72:0/4278295186 conn(0x7f39980ccd80 msgr2=0x7f39980cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:39.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.438+0000 7f399d462700 1 -- 172.21.15.72:0/4278295186 shutdown_connections 2023-03-24T23:47:39.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.438+0000 7f399d462700 1 -- 172.21.15.72:0/4278295186 wait complete. 2023-03-24T23:47:39.444 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.439+0000 7f399d462700 1 Processor -- start 2023-03-24T23:47:39.444 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.439+0000 7f399d462700 1 -- start start 2023-03-24T23:47:39.444 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.440+0000 7f399d462700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f399806d4a0 0x7f399815ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:39.444 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.440+0000 7f399d462700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f399806dae0 0x7f39980b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:39.444 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.440+0000 7f399d462700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f39980cad50 0x7f3998155e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:39.444 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.440+0000 7f399d462700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39980b5690 con 0x7f399806d4a0 2023-03-24T23:47:39.444 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.440+0000 7f399d462700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399815d220 con 0x7f399806dae0 2023-03-24T23:47:39.445 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.440+0000 7f399d462700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f399815d500 con 0x7f39980cad50 2023-03-24T23:47:39.445 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.440+0000 7f3996ffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f399806d4a0 0x7f399815ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:39.445 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.440+0000 7f3996ffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f399806d4a0 0x7f399815ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37828/0 (socket says 172.21.15.72:37828) 2023-03-24T23:47:39.445 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.440+0000 7f3996ffd700 1 -- 172.21.15.72:0/849814563 learned_addr learned my addr 172.21.15.72:0/849814563 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-03-24T23:47:39.445 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.440+0000 7f39967fc700 1 --2- 172.21.15.72:0/849814563 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f399806dae0 0x7f39980b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:39.445 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.440+0000 7f39977fe700 1 --2- 172.21.15.72:0/849814563 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f39980cad50 0x7f3998155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:39.445 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.440+0000 7f3996ffd700 1 -- 172.21.15.72:0/849814563 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f39980cad50 msgr2=0x7f3998155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:39.446 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.440+0000 7f3996ffd700 1 --2- 172.21.15.72:0/849814563 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f39980cad50 0x7f3998155e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:39.446 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.440+0000 7f3996ffd700 1 -- 172.21.15.72:0/849814563 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f399806dae0 msgr2=0x7f39980b4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:39.446 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.440+0000 7f3996ffd700 1 --2- 172.21.15.72:0/849814563 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f399806dae0 0x7f39980b4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:39.446 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.440+0000 7f3996ffd700 1 -- 172.21.15.72:0/849814563 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3990009b30 con 0x7f399806d4a0 2023-03-24T23:47:39.446 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.440+0000 7f3996ffd700 1 --2- 172.21.15.72:0/849814563 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f399806d4a0 0x7f399815ac60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f398c00c4d0 tx=0x7f398c00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:39.446 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.440+0000 7f397f7fe700 1 -- 172.21.15.72:0/849814563 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c011790 con 0x7f399806d4a0 2023-03-24T23:47:39.446 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.440+0000 7f397f7fe700 1 -- 172.21.15.72:0/849814563 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398c0118f0 con 0x7f399806d4a0 2023-03-24T23:47:39.446 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.440+0000 7f399d462700 1 -- 172.21.15.72:0/849814563 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f399806cca0 con 0x7f399806d4a0 2023-03-24T23:47:39.447 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.440+0000 7f397f7fe700 1 -- 172.21.15.72:0/849814563 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c011ca0 con 0x7f399806d4a0 2023-03-24T23:47:39.447 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.440+0000 7f399d462700 1 -- 172.21.15.72:0/849814563 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f399806c9c0 con 0x7f399806d4a0 2023-03-24T23:47:39.447 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.441+0000 7f399d462700 1 -- 172.21.15.72:0/849814563 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39980cfd80 con 0x7f399806d4a0 2023-03-24T23:47:39.447 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.442+0000 7f397f7fe700 1 -- 172.21.15.72:0/849814563 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f398c01c460 con 0x7f399806d4a0 2023-03-24T23:47:39.447 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.442+0000 7f397f7fe700 1 --2- 172.21.15.72:0/849814563 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f3984069de0 0x7f398406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:39.447 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.442+0000 7f397f7fe700 1 -- 172.21.15.72:0/849814563 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f398c0879b0 con 0x7f399806d4a0 2023-03-24T23:47:39.447 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.442+0000 7f39967fc700 1 --2- 172.21.15.72:0/849814563 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f3984069de0 0x7f398406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:39.448 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.443+0000 7f39967fc700 1 --2- 172.21.15.72:0/849814563 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f3984069de0 0x7f398406c290 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3998156e00 tx=0x7f3988008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:39.448 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.447+0000 7f397f7fe700 1 -- 172.21.15.72:0/849814563 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f398c053550 con 0x7f399806d4a0 2023-03-24T23:47:39.633 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.632+0000 7f399d462700 1 -- 172.21.15.72:0/849814563 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f399804e850 con 0x7f399806d4a0 2023-03-24T23:47:39.634 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.632+0000 7f397f7fe700 1 -- 172.21.15.72:0/849814563 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7f398c056b70 con 0x7f399806d4a0 2023-03-24T23:47:39.634 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 8 2023-03-24T23:47:39.636 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.634+0000 7f399d462700 1 -- 172.21.15.72:0/849814563 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f3984069de0 msgr2=0x7f398406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:39.636 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.634+0000 7f399d462700 1 --2- 172.21.15.72:0/849814563 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f3984069de0 0x7f398406c290 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3998156e00 tx=0x7f3988008040 comp rx=0 tx=0).stop 2023-03-24T23:47:39.636 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.634+0000 7f399d462700 1 -- 172.21.15.72:0/849814563 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f399806d4a0 msgr2=0x7f399815ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:39.636 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.634+0000 7f399d462700 1 --2- 172.21.15.72:0/849814563 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f399806d4a0 0x7f399815ac60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f398c00c4d0 tx=0x7f398c00c7e0 comp rx=0 tx=0).stop 2023-03-24T23:47:39.636 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.634+0000 7f399d462700 1 -- 172.21.15.72:0/849814563 shutdown_connections 2023-03-24T23:47:39.636 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.634+0000 7f399d462700 1 --2- 172.21.15.72:0/849814563 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f39980cad50 0x7f3998155e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:39.637 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.634+0000 7f399d462700 1 --2- 172.21.15.72:0/849814563 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f399806d4a0 0x7f399815ac60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:39.637 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.634+0000 7f399d462700 1 --2- 172.21.15.72:0/849814563 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f3984069de0 0x7f398406c290 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:39.637 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.634+0000 7f399d462700 1 --2- 172.21.15.72:0/849814563 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f399806dae0 0x7f39980b4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:39.637 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.634+0000 7f399d462700 1 -- 172.21.15.72:0/849814563 >> 172.21.15.72:0/849814563 conn(0x7f39980ccd80 msgr2=0x7f39980b0820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:39.637 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.634+0000 7f399d462700 1 -- 172.21.15.72:0/849814563 shutdown_connections 2023-03-24T23:47:39.637 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.635+0000 7f399d462700 1 -- 172.21.15.72:0/849814563 wait complete. 2023-03-24T23:47:39.647 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-03-24T23:47:39.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.770+0000 7f0f623ae700 1 Processor -- start 2023-03-24T23:47:39.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.770+0000 7f0f623ae700 1 -- start start 2023-03-24T23:47:39.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.771+0000 7f0f623ae700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f0f5c071af0 0x7f0f5c06b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:39.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.771+0000 7f0f623ae700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0f5c06baf0 0x7f0f5c0ca610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:39.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.771+0000 7f0f623ae700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f5c15d370 con 0x7f0f5c159970 2023-03-24T23:47:39.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.771+0000 7f0f623ae700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f5c06bf70 con 0x7f0f5c071af0 2023-03-24T23:47:39.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.771+0000 7f0f623ae700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f5c15d650 con 0x7f0f5c06baf0 2023-03-24T23:47:39.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.771+0000 7f0f5b7fe700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f0f5c071af0 0x7f0f5c06b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:39.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.771+0000 7f0f5b7fe700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f0f5c071af0 0x7f0f5c06b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37838/0 (socket says 172.21.15.72:37838) 2023-03-24T23:47:39.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.771+0000 7f0f5b7fe700 1 -- 172.21.15.72:0/1213462810 learned_addr learned my addr 172.21.15.72:0/1213462810 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-03-24T23:47:39.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.772+0000 7f0f6094b700 1 --2- 172.21.15.72:0/1213462810 >> v2:172.21.15.146:3300/0 conn(0x7f0f5c06baf0 0x7f0f5c0ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:39.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.772+0000 7f0f5b7fe700 1 -- 172.21.15.72:0/1213462810 >> v2:172.21.15.146:3300/0 conn(0x7f0f5c06baf0 msgr2=0x7f0f5c0ca610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T23:47:39.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.772+0000 7f0f5b7fe700 1 --2- 172.21.15.72:0/1213462810 >> v2:172.21.15.146:3300/0 conn(0x7f0f5c06baf0 0x7f0f5c0ca610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:39.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.772+0000 7f0f5b7fe700 1 -- 172.21.15.72:0/1213462810 >> v1:172.21.15.146:6789/0 conn(0x7f0f5c159970 legacy=0x7f0f5c15dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T23:47:39.776 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.772+0000 7f0f5b7fe700 1 -- 172.21.15.72:0/1213462810 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5c071f40 con 0x7f0f5c071af0 2023-03-24T23:47:39.776 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.772+0000 7f0f5b7fe700 1 --2- 172.21.15.72:0/1213462810 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0f5c071af0 0x7f0f5c06b5b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c06f2e0 tx=0x7f0f4400be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf87305c2ae1393 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:39.776 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.772+0000 7f0f5affd700 1 -- 172.21.15.72:0/1213462810 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4400dcf0 con 0x7f0f5c071af0 2023-03-24T23:47:39.776 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.772+0000 7f0f5affd700 1 -- 172.21.15.72:0/1213462810 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f4400de50 con 0x7f0f5c071af0 2023-03-24T23:47:39.776 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.772+0000 7f0f5affd700 1 -- 172.21.15.72:0/1213462810 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f44004250 con 0x7f0f5c071af0 2023-03-24T23:47:39.776 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.772+0000 7f0f623ae700 1 -- 172.21.15.72:0/1213462810 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0f5c071af0 msgr2=0x7f0f5c06b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:39.776 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.772+0000 7f0f623ae700 1 --2- 172.21.15.72:0/1213462810 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0f5c071af0 0x7f0f5c06b5b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c06f2e0 tx=0x7f0f4400be70 comp rx=0 tx=0).stop 2023-03-24T23:47:39.777 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.772+0000 7f0f623ae700 1 -- 172.21.15.72:0/1213462810 shutdown_connections 2023-03-24T23:47:39.777 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.772+0000 7f0f623ae700 1 --2- 172.21.15.72:0/1213462810 >> v2:172.21.15.146:3300/0 conn(0x7f0f5c06baf0 0x7f0f5c0ca610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:39.777 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.772+0000 7f0f623ae700 1 --2- 172.21.15.72:0/1213462810 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0f5c071af0 0x7f0f5c06b5b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:39.777 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.772+0000 7f0f623ae700 1 -- 172.21.15.72:0/1213462810 >> 172.21.15.72:0/1213462810 conn(0x7f0f5c0cfe80 msgr2=0x7f0f5c0d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:39.777 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.773+0000 7f0f623ae700 1 -- 172.21.15.72:0/1213462810 shutdown_connections 2023-03-24T23:47:39.777 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.773+0000 7f0f623ae700 1 -- 172.21.15.72:0/1213462810 wait complete. 2023-03-24T23:47:39.779 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.774+0000 7f0f623ae700 1 Processor -- start 2023-03-24T23:47:39.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.774+0000 7f0f623ae700 1 -- start start 2023-03-24T23:47:39.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.775+0000 7f0f623ae700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f5c06baf0 0x7f0f5c1edf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:39.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.775+0000 7f0f623ae700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f5c071af0 0x7f0f5c0af900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:39.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.775+0000 7f0f623ae700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0f5c159970 0x7f0f5c14c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:39.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.775+0000 7f0f623ae700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c0b0150 con 0x7f0f5c159970 2023-03-24T23:47:39.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.775+0000 7f0f623ae700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c0b0430 con 0x7f0f5c06baf0 2023-03-24T23:47:39.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.775+0000 7f0f623ae700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c0b0710 con 0x7f0f5c071af0 2023-03-24T23:47:39.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.775+0000 7f0f6094b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0f5c159970 0x7f0f5c14c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:39.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.775+0000 7f0f6094b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0f5c159970 0x7f0f5c14c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37850/0 (socket says 172.21.15.72:37850) 2023-03-24T23:47:39.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.775+0000 7f0f6094b700 1 -- 172.21.15.72:0/402997987 learned_addr learned my addr 172.21.15.72:0/402997987 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-03-24T23:47:39.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.775+0000 7f0f5bfff700 1 --2- 172.21.15.72:0/402997987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f5c06baf0 0x7f0f5c1edf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:39.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.775+0000 7f0f5b7fe700 1 --2- 172.21.15.72:0/402997987 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f5c071af0 0x7f0f5c0af900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:39.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.775+0000 7f0f6094b700 1 -- 172.21.15.72:0/402997987 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f5c071af0 msgr2=0x7f0f5c0af900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:39.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.775+0000 7f0f6094b700 1 --2- 172.21.15.72:0/402997987 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f5c071af0 0x7f0f5c0af900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:39.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.775+0000 7f0f6094b700 1 -- 172.21.15.72:0/402997987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f5c06baf0 msgr2=0x7f0f5c1edf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:39.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.775+0000 7f0f6094b700 1 --2- 172.21.15.72:0/402997987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f5c06baf0 0x7f0f5c1edf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:39.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.775+0000 7f0f6094b700 1 -- 172.21.15.72:0/402997987 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f4400b7f0 con 0x7f0f5c159970 2023-03-24T23:47:39.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.775+0000 7f0f6094b700 1 --2- 172.21.15.72:0/402997987 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0f5c159970 0x7f0f5c14c250 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0f54007e40 tx=0x7f0f5400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:39.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.775+0000 7f0f58ff9700 1 -- 172.21.15.72:0/402997987 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5400ed30 con 0x7f0f5c159970 2023-03-24T23:47:39.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.775+0000 7f0f58ff9700 1 -- 172.21.15.72:0/402997987 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f54014420 con 0x7f0f5c159970 2023-03-24T23:47:39.783 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.775+0000 7f0f58ff9700 1 -- 172.21.15.72:0/402997987 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f540147d0 con 0x7f0f5c159970 2023-03-24T23:47:39.783 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.775+0000 7f0f623ae700 1 -- 172.21.15.72:0/402997987 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f5c06dad0 con 0x7f0f5c159970 2023-03-24T23:47:39.783 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.775+0000 7f0f623ae700 1 -- 172.21.15.72:0/402997987 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f5c06d7f0 con 0x7f0f5c159970 2023-03-24T23:47:39.783 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.776+0000 7f0f623ae700 1 -- 172.21.15.72:0/402997987 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f5c06ddb0 con 0x7f0f5c159970 2023-03-24T23:47:39.783 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.777+0000 7f0f58ff9700 1 -- 172.21.15.72:0/402997987 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f0f54014930 con 0x7f0f5c159970 2023-03-24T23:47:39.783 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.777+0000 7f0f58ff9700 1 --2- 172.21.15.72:0/402997987 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f0f3c069de0 0x7f0f3c06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:39.783 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.777+0000 7f0f58ff9700 1 -- 172.21.15.72:0/402997987 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0f54088370 con 0x7f0f5c159970 2023-03-24T23:47:39.784 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.777+0000 7f0f5bfff700 1 --2- 172.21.15.72:0/402997987 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f0f3c069de0 0x7f0f3c06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:39.784 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.778+0000 7f0f5bfff700 1 --2- 172.21.15.72:0/402997987 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f0f3c069de0 0x7f0f3c06c290 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c152550 tx=0x7f0f4c0090e0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:39.784 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:39.781+0000 7f0f58ff9700 1 -- 172.21.15.72:0/402997987 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f0f54053fa0 con 0x7f0f5c159970 2023-03-24T23:47:40.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.007+0000 7f0f623ae700 1 -- 172.21.15.72:0/402997987 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0f5c0d0c00 con 0x7f0f5c159970 2023-03-24T23:47:40.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.008+0000 7f0f58ff9700 1 -- 172.21.15.72:0/402997987 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3530 (secure 0 0 0) 0x7f0f540575c0 con 0x7f0f5c159970 2023-03-24T23:47:40.009 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:47:40.010 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1050596002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1050596002},{"type":"v1","addr":"172.21.15.146:6835","nonce":1050596002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/2462302384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":2462302384},{"type":"v1","addr":"172.21.15.72:6835","nonce":2462302384}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2874805042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":2874805042},{"type":"v1","addr":"172.21.15.72:6837","nonce":2874805042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4312,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3542244818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3542244818},{"type":"v1","addr":"172.21.15.146:6833","nonce":3542244818}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-03-24T23:47:40.011 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.010+0000 7f0f623ae700 1 -- 172.21.15.72:0/402997987 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f0f3c069de0 msgr2=0x7f0f3c06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:40.011 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.010+0000 7f0f623ae700 1 --2- 172.21.15.72:0/402997987 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f0f3c069de0 0x7f0f3c06c290 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c152550 tx=0x7f0f4c0090e0 comp rx=0 tx=0).stop 2023-03-24T23:47:40.011 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.010+0000 7f0f623ae700 1 -- 172.21.15.72:0/402997987 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0f5c159970 msgr2=0x7f0f5c14c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:40.011 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.010+0000 7f0f623ae700 1 --2- 172.21.15.72:0/402997987 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0f5c159970 0x7f0f5c14c250 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0f54007e40 tx=0x7f0f5400e3c0 comp rx=0 tx=0).stop 2023-03-24T23:47:40.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.010+0000 7f0f623ae700 1 -- 172.21.15.72:0/402997987 shutdown_connections 2023-03-24T23:47:40.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.010+0000 7f0f623ae700 1 --2- 172.21.15.72:0/402997987 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0f5c159970 0x7f0f5c14c250 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:40.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.010+0000 7f0f623ae700 1 --2- 172.21.15.72:0/402997987 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f0f3c069de0 0x7f0f3c06c290 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:40.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.010+0000 7f0f623ae700 1 --2- 172.21.15.72:0/402997987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f5c06baf0 0x7f0f5c1edf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:40.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.010+0000 7f0f623ae700 1 --2- 172.21.15.72:0/402997987 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f5c071af0 0x7f0f5c0af900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:40.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.010+0000 7f0f623ae700 1 -- 172.21.15.72:0/402997987 >> 172.21.15.72:0/402997987 conn(0x7f0f5c0cfe80 msgr2=0x7f0f5c152c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:40.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.010+0000 7f0f623ae700 1 -- 172.21.15.72:0/402997987 shutdown_connections 2023-03-24T23:47:40.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.010+0000 7f0f623ae700 1 -- 172.21.15.72:0/402997987 wait complete. 2023-03-24T23:47:40.013 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 3 2023-03-24T23:47:40.023 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-24T23:47:40.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.143+0000 7f4abbd41700 1 Processor -- start 2023-03-24T23:47:40.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.143+0000 7f4abbd41700 1 -- start start 2023-03-24T23:47:40.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.144+0000 7f4abbd41700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f4ab40cff20 0x7f4ab40d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:40.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.144+0000 7f4abbd41700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ab415bb60 con 0x7f4ab406faf0 2023-03-24T23:47:40.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.144+0000 7f4abbd41700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ab40b37a0 con 0x7f4ab40cd980 2023-03-24T23:47:40.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.144+0000 7f4abbd41700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ab40d0830 con 0x7f4ab40cff20 2023-03-24T23:47:40.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.144+0000 7f4ab9add700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f4ab40cff20 0x7f4ab40d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:40.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.144+0000 7f4ab92dc700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f4ab406faf0 0x7f4ab40b2980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:50022/0 (socket says 172.21.15.72:50022) 2023-03-24T23:47:40.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.144+0000 7f4ab92dc700 1 -- 172.21.15.72:0/2558093030 learned_addr learned my addr 172.21.15.72:0/2558093030 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-03-24T23:47:40.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.144+0000 7f4ab9add700 1 -- 172.21.15.72:0/2558093030 >> v1:172.21.15.146:6789/0 conn(0x7f4ab40cd980 legacy=0x7f4ab40b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T23:47:40.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.144+0000 7f4ab9add700 1 -- 172.21.15.72:0/2558093030 >> v1:172.21.15.72:6789/0 conn(0x7f4ab406faf0 legacy=0x7f4ab40b2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:40.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.144+0000 7f4ab9add700 1 -- 172.21.15.72:0/2558093030 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab415ac70 con 0x7f4ab40cff20 2023-03-24T23:47:40.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.144+0000 7f4ab8adb700 1 -- 172.21.15.72:0/2558093030 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 424849542 0 0) 0x7f4ab415bb60 con 0x7f4ab406faf0 2023-03-24T23:47:40.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.145+0000 7f4ab9add700 1 --2- 172.21.15.72:0/2558093030 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ab40cff20 0x7f4ab40d02f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4009840 tx=0x7f4aa4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d36811a75d292d6 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:40.148 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.145+0000 7f4ab8adb700 1 -- 172.21.15.72:0/2558093030 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4004230 con 0x7f4ab40cff20 2023-03-24T23:47:40.148 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.145+0000 7f4ab8adb700 1 -- 172.21.15.72:0/2558093030 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa4004390 con 0x7f4ab40cff20 2023-03-24T23:47:40.148 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.145+0000 7f4ab8adb700 1 -- 172.21.15.72:0/2558093030 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa40036a0 con 0x7f4ab40cff20 2023-03-24T23:47:40.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.145+0000 7f4abbd41700 1 -- 172.21.15.72:0/2558093030 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ab40cff20 msgr2=0x7f4ab40d02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:40.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.145+0000 7f4abbd41700 1 --2- 172.21.15.72:0/2558093030 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ab40cff20 0x7f4ab40d02f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4009840 tx=0x7f4aa4009b50 comp rx=0 tx=0).stop 2023-03-24T23:47:40.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.145+0000 7f4abbd41700 1 -- 172.21.15.72:0/2558093030 shutdown_connections 2023-03-24T23:47:40.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.145+0000 7f4abbd41700 1 --2- 172.21.15.72:0/2558093030 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ab40cff20 0x7f4ab40d02f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:40.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.145+0000 7f4abbd41700 1 -- 172.21.15.72:0/2558093030 >> 172.21.15.72:0/2558093030 conn(0x7f4ab406e360 msgr2=0x7f4ab406e730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:40.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.145+0000 7f4abbd41700 1 -- 172.21.15.72:0/2558093030 shutdown_connections 2023-03-24T23:47:40.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.145+0000 7f4abbd41700 1 -- 172.21.15.72:0/2558093030 wait complete. 2023-03-24T23:47:40.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.147+0000 7f4abbd41700 1 Processor -- start 2023-03-24T23:47:40.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.147+0000 7f4abbd41700 1 -- start start 2023-03-24T23:47:40.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.147+0000 7f4abbd41700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ab406faf0 0x7f4ab41eff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:40.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.147+0000 7f4abbd41700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ab40cd980 0x7f4ab41f3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:40.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.147+0000 7f4abbd41700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ab40cff20 0x7f4ab40b65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:40.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.147+0000 7f4abbd41700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab4146bb0 con 0x7f4ab40cff20 2023-03-24T23:47:40.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.147+0000 7f4abbd41700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab40b4080 con 0x7f4ab40cd980 2023-03-24T23:47:40.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.147+0000 7f4abbd41700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab41f05b0 con 0x7f4ab406faf0 2023-03-24T23:47:40.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.147+0000 7f4aba2de700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ab40cff20 0x7f4ab40b65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:40.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.147+0000 7f4aba2de700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ab40cff20 0x7f4ab40b65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54434/0 (socket says 172.21.15.72:54434) 2023-03-24T23:47:40.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.147+0000 7f4aba2de700 1 -- 172.21.15.72:0/2347578074 learned_addr learned my addr 172.21.15.72:0/2347578074 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-03-24T23:47:40.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.147+0000 7f4ab92dc700 1 --2- 172.21.15.72:0/2347578074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ab40cd980 0x7f4ab41f3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:40.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.147+0000 7f4ab9add700 1 --2- 172.21.15.72:0/2347578074 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ab406faf0 0x7f4ab41eff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:40.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.148+0000 7f4aba2de700 1 -- 172.21.15.72:0/2347578074 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ab406faf0 msgr2=0x7f4ab41eff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:40.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.148+0000 7f4aba2de700 1 --2- 172.21.15.72:0/2347578074 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ab406faf0 0x7f4ab41eff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:40.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.148+0000 7f4aba2de700 1 -- 172.21.15.72:0/2347578074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ab40cd980 msgr2=0x7f4ab41f3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:40.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.148+0000 7f4aba2de700 1 --2- 172.21.15.72:0/2347578074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ab40cd980 0x7f4ab41f3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:40.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.148+0000 7f4aba2de700 1 -- 172.21.15.72:0/2347578074 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa40094a0 con 0x7f4ab40cff20 2023-03-24T23:47:40.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.148+0000 7f4ab9add700 1 --2- 172.21.15.72:0/2347578074 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ab406faf0 0x7f4ab41eff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-24T23:47:40.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.148+0000 7f4aba2de700 1 --2- 172.21.15.72:0/2347578074 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ab40cff20 0x7f4ab40b65b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4aac00cc10 tx=0x7f4aac00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:40.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.148+0000 7f4aaa7fc700 1 -- 172.21.15.72:0/2347578074 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aac00ab10 con 0x7f4ab40cff20 2023-03-24T23:47:40.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.148+0000 7f4aaa7fc700 1 -- 172.21.15.72:0/2347578074 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aac00ac70 con 0x7f4ab40cff20 2023-03-24T23:47:40.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.148+0000 7f4aaa7fc700 1 -- 172.21.15.72:0/2347578074 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aac012420 con 0x7f4ab40cff20 2023-03-24T23:47:40.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.148+0000 7f4abbd41700 1 -- 172.21.15.72:0/2347578074 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ab415b230 con 0x7f4ab40cff20 2023-03-24T23:47:40.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.148+0000 7f4abbd41700 1 -- 172.21.15.72:0/2347578074 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ab415af50 con 0x7f4ab40cff20 2023-03-24T23:47:40.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.149+0000 7f4abbd41700 1 -- 172.21.15.72:0/2347578074 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ab415b510 con 0x7f4ab40cff20 2023-03-24T23:47:40.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.149+0000 7f4aaa7fc700 1 -- 172.21.15.72:0/2347578074 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f4aac012580 con 0x7f4ab40cff20 2023-03-24T23:47:40.156 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.150+0000 7f4aaa7fc700 1 --2- 172.21.15.72:0/2347578074 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f4a98069de0 0x7f4a9806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:40.156 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.150+0000 7f4aaa7fc700 1 -- 172.21.15.72:0/2347578074 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f4aac087fb0 con 0x7f4ab40cff20 2023-03-24T23:47:40.156 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.150+0000 7f4ab9add700 1 --2- 172.21.15.72:0/2347578074 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f4a98069de0 0x7f4a9806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:40.156 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.150+0000 7f4ab9add700 1 --2- 172.21.15.72:0/2347578074 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f4a98069de0 0x7f4a9806c290 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4000c00 tx=0x7f4aa401c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:40.156 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.154+0000 7f4aaa7fc700 1 -- 172.21.15.72:0/2347578074 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f4aac053bb0 con 0x7f4ab40cff20 2023-03-24T23:47:40.342 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.341+0000 7f4abbd41700 1 -- 172.21.15.72:0/2347578074 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ab404e850 con 0x7f4ab40cff20 2023-03-24T23:47:40.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.342+0000 7f4aaa7fc700 1 -- 172.21.15.72:0/2347578074 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+14264 (secure 0 0 0) 0x7f4aac0571d0 con 0x7f4ab40cff20 2023-03-24T23:47:40.343 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:47:40.344 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":19,"fsid":"d3c122b7-f0b4-4c96-8f85-86f129e51880","created":"2023-03-24T23:47:18.348997+0000","modified":"2023-03-24T23:47:39.603658+0000","last_up_change":"2023-03-24T23:47:29.397334+0000","last_in_change":"2023-03-24T23:47:20.920082+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-24T23:47:30.335434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-24T23:47:36.817567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"993abb78-a510-4d76-af32-9015a2b3e5d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":114934},{"type":"v1","addr":"172.21.15.72:6809","nonce":114934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":114934},{"type":"v1","addr":"172.21.15.72:6811","nonce":114934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":114934},{"type":"v1","addr":"172.21.15.72:6815","nonce":114934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":114934},{"type":"v1","addr":"172.21.15.72:6813","nonce":114934}]},"public_addr":"172.21.15.72:6809/114934","cluster_addr":"172.21.15.72:6811/114934","heartbeat_back_addr":"172.21.15.72:6815/114934","heartbeat_front_addr":"172.21.15.72:6813/114934","state":["exists","up"]},{"osd":1,"uuid":"ddfa95f0-f08c-4dce-b961-2e3a9f4bcb3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":114933},{"type":"v1","addr":"172.21.15.72:6801","nonce":114933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":114933},{"type":"v1","addr":"172.21.15.72:6803","nonce":114933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":114933},{"type":"v1","addr":"172.21.15.72:6807","nonce":114933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":114933},{"type":"v1","addr":"172.21.15.72:6805","nonce":114933}]},"public_addr":"172.21.15.72:6801/114933","cluster_addr":"172.21.15.72:6803/114933","heartbeat_back_addr":"172.21.15.72:6807/114933","heartbeat_front_addr":"172.21.15.72:6805/114933","state":["exists","up"]},{"osd":2,"uuid":"cfcf995d-5a1f-47c1-9691-858f305b1f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":114943},{"type":"v1","addr":"172.21.15.72:6818","nonce":114943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":114943},{"type":"v1","addr":"172.21.15.72:6822","nonce":114943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":114943},{"type":"v1","addr":"172.21.15.72:6830","nonce":114943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":114943},{"type":"v1","addr":"172.21.15.72:6826","nonce":114943}]},"public_addr":"172.21.15.72:6818/114943","cluster_addr":"172.21.15.72:6822/114943","heartbeat_back_addr":"172.21.15.72:6830/114943","heartbeat_front_addr":"172.21.15.72:6826/114943","state":["exists","up"]},{"osd":3,"uuid":"17c7d98c-55ba-4001-8eac-475f82cc31e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6817","nonce":114946},{"type":"v1","addr":"172.21.15.72:6819","nonce":114946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6821","nonce":114946},{"type":"v1","addr":"172.21.15.72:6823","nonce":114946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6829","nonce":114946},{"type":"v1","addr":"172.21.15.72:6831","nonce":114946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6825","nonce":114946},{"type":"v1","addr":"172.21.15.72:6827","nonce":114946}]},"public_addr":"172.21.15.72:6819/114946","cluster_addr":"172.21.15.72:6823/114946","heartbeat_back_addr":"172.21.15.72:6831/114946","heartbeat_front_addr":"172.21.15.72:6827/114946","state":["exists","up"]},{"osd":4,"uuid":"c78725a3-3c83-4119-a657-b19b187170e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":115192},{"type":"v1","addr":"172.21.15.146:6817","nonce":115192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":115192},{"type":"v1","addr":"172.21.15.146:6819","nonce":115192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":115192},{"type":"v1","addr":"172.21.15.146:6823","nonce":115192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":115192},{"type":"v1","addr":"172.21.15.146:6821","nonce":115192}]},"public_addr":"172.21.15.146:6817/115192","cluster_addr":"172.21.15.146:6819/115192","heartbeat_back_addr":"172.21.15.146:6823/115192","heartbeat_front_addr":"172.21.15.146:6821/115192","state":["exists","up"]},{"osd":5,"uuid":"151bcb4f-de54-4e07-be8b-d212f724aec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":115190},{"type":"v1","addr":"172.21.15.146:6801","nonce":115190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":115190},{"type":"v1","addr":"172.21.15.146:6803","nonce":115190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":115190},{"type":"v1","addr":"172.21.15.146:6807","nonce":115190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":115190},{"type":"v1","addr":"172.21.15.146:6805","nonce":115190}]},"public_addr":"172.21.15.146:6801/115190","cluster_addr":"172.21.15.146:6803/115190","heartbeat_back_addr":"172.21.15.146:6807/115190","heartbeat_front_addr":"172.21.15.146:6805/115190","state":["exists","up"]},{"osd":6,"uuid":"e5373fa9-b440-4c05-aa55-1618cc42c18d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":115191},{"type":"v1","addr":"172.21.15.146:6809","nonce":115191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":115191},{"type":"v1","addr":"172.21.15.146:6811","nonce":115191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":115191},{"type":"v1","addr":"172.21.15.146:6815","nonce":115191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":115191},{"type":"v1","addr":"172.21.15.146:6813","nonce":115191}]},"public_addr":"172.21.15.146:6809/115191","cluster_addr":"172.21.15.146:6811/115191","heartbeat_back_addr":"172.21.15.146:6815/115191","heartbeat_front_addr":"172.21.15.146:6813/115191","state":["exists","up"]},{"osd":7,"uuid":"1a67b0d5-8adc-4dd7-825c-6f26334a9a6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":115221},{"type":"v1","addr":"172.21.15.146:6825","nonce":115221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":115221},{"type":"v1","addr":"172.21.15.146:6827","nonce":115221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":115221},{"type":"v1","addr":"172.21.15.146:6831","nonce":115221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":115221},{"type":"v1","addr":"172.21.15.146:6829","nonce":115221}]},"public_addr":"172.21.15.146:6825/115221","cluster_addr":"172.21.15.146:6827/115221","heartbeat_back_addr":"172.21.15.146:6831/115221","heartbeat_front_addr":"172.21.15.146:6829/115221","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T23:47:26.063902+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T23:47:26.140546+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T23:47:26.037678+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T23:47:26.144856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T23:47:26.078748+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T23:47:26.005133+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T23:47:26.071184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T23:47:26.089707+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-24T23:47:40.345 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.344+0000 7f4abbd41700 1 -- 172.21.15.72:0/2347578074 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f4a98069de0 msgr2=0x7f4a9806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:40.345 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.344+0000 7f4abbd41700 1 --2- 172.21.15.72:0/2347578074 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f4a98069de0 0x7f4a9806c290 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4000c00 tx=0x7f4aa401c040 comp rx=0 tx=0).stop 2023-03-24T23:47:40.346 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.344+0000 7f4abbd41700 1 -- 172.21.15.72:0/2347578074 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ab40cff20 msgr2=0x7f4ab40b65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:40.346 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.344+0000 7f4abbd41700 1 --2- 172.21.15.72:0/2347578074 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ab40cff20 0x7f4ab40b65b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4aac00cc10 tx=0x7f4aac00cf20 comp rx=0 tx=0).stop 2023-03-24T23:47:40.346 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.344+0000 7f4abbd41700 1 -- 172.21.15.72:0/2347578074 shutdown_connections 2023-03-24T23:47:40.346 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.344+0000 7f4abbd41700 1 --2- 172.21.15.72:0/2347578074 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ab40cff20 0x7f4ab40b65b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:40.346 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.344+0000 7f4abbd41700 1 --2- 172.21.15.72:0/2347578074 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ab406faf0 0x7f4ab41eff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:40.346 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.344+0000 7f4abbd41700 1 --2- 172.21.15.72:0/2347578074 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f4a98069de0 0x7f4a9806c290 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:40.346 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.344+0000 7f4abbd41700 1 --2- 172.21.15.72:0/2347578074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ab40cd980 0x7f4ab41f3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:40.347 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.344+0000 7f4abbd41700 1 -- 172.21.15.72:0/2347578074 >> 172.21.15.72:0/2347578074 conn(0x7f4ab406e360 msgr2=0x7f4ab406e730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:40.347 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.344+0000 7f4abbd41700 1 -- 172.21.15.72:0/2347578074 shutdown_connections 2023-03-24T23:47:40.347 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.344+0000 7f4abbd41700 1 -- 172.21.15.72:0/2347578074 wait complete. 2023-03-24T23:47:40.358 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-03-24T23:47:30.335434+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-03-24T23:47:36.817567+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-03-24T23:47:40.359 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-24T23:47:40.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.483+0000 7fd42e833700 1 Processor -- start 2023-03-24T23:47:40.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.483+0000 7fd42e833700 1 -- start start 2023-03-24T23:47:40.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.483+0000 7fd42e833700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd4280b23c0 0x7fd4280b5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:40.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.483+0000 7fd42e833700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd4280d1710 0x7fd4280cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:40.488 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.483+0000 7fd42e833700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4280d2830 con 0x7fd4280d1b60 2023-03-24T23:47:40.488 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.483+0000 7fd42e833700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4280b2790 con 0x7fd4280b23c0 2023-03-24T23:47:40.488 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.484+0000 7fd42e833700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4280d2df0 con 0x7fd4280d1710 2023-03-24T23:47:40.488 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.484+0000 7fd427fff700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd4280b23c0 0x7fd4280b5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:40.488 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.484+0000 7fd427fff700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd4280b23c0 0x7fd4280b5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54436/0 (socket says 172.21.15.72:54436) 2023-03-24T23:47:40.488 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.484+0000 7fd427fff700 1 -- 172.21.15.72:0/735023336 learned_addr learned my addr 172.21.15.72:0/735023336 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-03-24T23:47:40.488 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.484+0000 7fd4277fe700 1 --2- 172.21.15.72:0/735023336 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd4280d1710 0x7fd4280cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:40.489 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.484+0000 7fd427fff700 1 -- 172.21.15.72:0/735023336 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd4280d1710 msgr2=0x7fd4280cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T23:47:40.489 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.484+0000 7fd427fff700 1 --2- 172.21.15.72:0/735023336 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd4280d1710 0x7fd4280cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:40.489 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.484+0000 7fd427fff700 1 -- 172.21.15.72:0/735023336 >> v1:172.21.15.146:6789/0 conn(0x7fd4280d1b60 legacy=0x7fd4280d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T23:47:40.489 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.484+0000 7fd427fff700 1 -- 172.21.15.72:0/735023336 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4280cd920 con 0x7fd4280b23c0 2023-03-24T23:47:40.489 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.484+0000 7fd427fff700 1 --2- 172.21.15.72:0/735023336 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4280b23c0 0x7fd4280b5f80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd4180098f0 tx=0x7fd418009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191d101621a2856b server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:40.489 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.484+0000 7fd426ffd700 1 -- 172.21.15.72:0/735023336 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41800b8a0 con 0x7fd4280b23c0 2023-03-24T23:47:40.489 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.484+0000 7fd426ffd700 1 -- 172.21.15.72:0/735023336 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd41800ba00 con 0x7fd4280b23c0 2023-03-24T23:47:40.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.484+0000 7fd426ffd700 1 -- 172.21.15.72:0/735023336 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41800bd90 con 0x7fd4280b23c0 2023-03-24T23:47:40.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.485+0000 7fd42e833700 1 -- 172.21.15.72:0/735023336 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4280b23c0 msgr2=0x7fd4280b5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:40.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.485+0000 7fd42e833700 1 --2- 172.21.15.72:0/735023336 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4280b23c0 0x7fd4280b5f80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd4180098f0 tx=0x7fd418009c00 comp rx=0 tx=0).stop 2023-03-24T23:47:40.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.485+0000 7fd42e833700 1 -- 172.21.15.72:0/735023336 shutdown_connections 2023-03-24T23:47:40.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.485+0000 7fd42e833700 1 --2- 172.21.15.72:0/735023336 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4280b23c0 0x7fd4280b5f80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:40.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.485+0000 7fd42e833700 1 --2- 172.21.15.72:0/735023336 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd4280d1710 0x7fd4280cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:40.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.485+0000 7fd42e833700 1 -- 172.21.15.72:0/735023336 >> 172.21.15.72:0/735023336 conn(0x7fd428066140 msgr2=0x7fd428066540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:40.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.485+0000 7fd42e833700 1 -- 172.21.15.72:0/735023336 shutdown_connections 2023-03-24T23:47:40.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.485+0000 7fd42e833700 1 -- 172.21.15.72:0/735023336 wait complete. 2023-03-24T23:47:40.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.486+0000 7fd42e833700 1 Processor -- start 2023-03-24T23:47:40.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.486+0000 7fd42e833700 1 -- start start 2023-03-24T23:47:40.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.487+0000 7fd42e833700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd4280b23c0 0x7fd4281f6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:40.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.487+0000 7fd42e833700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd4280d1710 0x7fd4280d0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:40.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.487+0000 7fd42e833700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4280ca200 0x7fd4280cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:40.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.487+0000 7fd42e833700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4281522b0 con 0x7fd4280ca200 2023-03-24T23:47:40.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.487+0000 7fd42e833700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4281f6c70 con 0x7fd4280b23c0 2023-03-24T23:47:40.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.487+0000 7fd42e833700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4281f5ce0 con 0x7fd4280d1710 2023-03-24T23:47:40.494 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.487+0000 7fd42cdd0700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4280ca200 0x7fd4280cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:40.494 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.487+0000 7fd42cdd0700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4280ca200 0x7fd4280cb2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54444/0 (socket says 172.21.15.72:54444) 2023-03-24T23:47:40.494 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.487+0000 7fd42cdd0700 1 -- 172.21.15.72:0/708478297 learned_addr learned my addr 172.21.15.72:0/708478297 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-03-24T23:47:40.494 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.487+0000 7fd427fff700 1 --2- 172.21.15.72:0/708478297 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd4280b23c0 0x7fd4281f6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:40.494 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.487+0000 7fd42cdd0700 1 -- 172.21.15.72:0/708478297 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd4280d1710 msgr2=0x7fd4280d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:40.494 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.487+0000 7fd42cdd0700 1 --2- 172.21.15.72:0/708478297 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd4280d1710 0x7fd4280d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:40.494 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.487+0000 7fd42cdd0700 1 -- 172.21.15.72:0/708478297 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd4280b23c0 msgr2=0x7fd4281f6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:40.494 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.487+0000 7fd42cdd0700 1 --2- 172.21.15.72:0/708478297 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd4280b23c0 0x7fd4281f6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:40.495 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.487+0000 7fd42cdd0700 1 -- 172.21.15.72:0/708478297 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd418009550 con 0x7fd4280ca200 2023-03-24T23:47:40.495 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.487+0000 7fd427fff700 1 --2- 172.21.15.72:0/708478297 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd4280b23c0 0x7fd4281f6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-24T23:47:40.495 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.487+0000 7fd42cdd0700 1 --2- 172.21.15.72:0/708478297 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4280ca200 0x7fd4280cb2b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd42000cbc0 tx=0x7fd42000cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:40.495 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.487+0000 7fd424ff9700 1 -- 172.21.15.72:0/708478297 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd420003c00 con 0x7fd4280ca200 2023-03-24T23:47:40.495 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.488+0000 7fd424ff9700 1 -- 172.21.15.72:0/708478297 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd420003d60 con 0x7fd4280ca200 2023-03-24T23:47:40.495 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.488+0000 7fd424ff9700 1 -- 172.21.15.72:0/708478297 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42000ab90 con 0x7fd4280ca200 2023-03-24T23:47:40.495 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.488+0000 7fd42e833700 1 -- 172.21.15.72:0/708478297 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4280d2a80 con 0x7fd4280ca200 2023-03-24T23:47:40.496 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.488+0000 7fd42e833700 1 -- 172.21.15.72:0/708478297 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4280cdc00 con 0x7fd4280ca200 2023-03-24T23:47:40.496 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.488+0000 7fd42e833700 1 -- 172.21.15.72:0/708478297 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4280cf530 con 0x7fd4280ca200 2023-03-24T23:47:40.496 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.489+0000 7fd424ff9700 1 -- 172.21.15.72:0/708478297 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fd42000acf0 con 0x7fd4280ca200 2023-03-24T23:47:40.496 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.489+0000 7fd424ff9700 1 --2- 172.21.15.72:0/708478297 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7fd414069de0 0x7fd41406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:40.496 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.489+0000 7fd427fff700 1 --2- 172.21.15.72:0/708478297 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7fd414069de0 0x7fd41406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:40.496 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.489+0000 7fd424ff9700 1 -- 172.21.15.72:0/708478297 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fd420087de0 con 0x7fd4280ca200 2023-03-24T23:47:40.496 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.490+0000 7fd427fff700 1 --2- 172.21.15.72:0/708478297 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7fd414069de0 0x7fd41406c290 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd418014040 tx=0x7fd418008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:40.497 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.494+0000 7fd424ff9700 1 -- 172.21.15.72:0/708478297 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fd420053950 con 0x7fd4280ca200 2023-03-24T23:47:40.678 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.677+0000 7fd42e833700 1 -- 172.21.15.72:0/708478297 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd42804e850 con 0x7fd4280ca200 2023-03-24T23:47:40.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.677+0000 7fd424ff9700 1 -- 172.21.15.72:0/708478297 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd420056f70 con 0x7fd4280ca200 2023-03-24T23:47:40.679 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2023-03-24T23:47:40.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.679+0000 7fd42e833700 1 -- 172.21.15.72:0/708478297 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7fd414069de0 msgr2=0x7fd41406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:40.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.679+0000 7fd42e833700 1 --2- 172.21.15.72:0/708478297 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7fd414069de0 0x7fd41406c290 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd418014040 tx=0x7fd418008030 comp rx=0 tx=0).stop 2023-03-24T23:47:40.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.679+0000 7fd42e833700 1 -- 172.21.15.72:0/708478297 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4280ca200 msgr2=0x7fd4280cb2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:40.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.679+0000 7fd42e833700 1 --2- 172.21.15.72:0/708478297 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4280ca200 0x7fd4280cb2b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd42000cbc0 tx=0x7fd42000cf80 comp rx=0 tx=0).stop 2023-03-24T23:47:40.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.680+0000 7fd42e833700 1 -- 172.21.15.72:0/708478297 shutdown_connections 2023-03-24T23:47:40.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.680+0000 7fd42e833700 1 --2- 172.21.15.72:0/708478297 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4280ca200 0x7fd4280cb2b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:40.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.680+0000 7fd42e833700 1 --2- 172.21.15.72:0/708478297 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7fd414069de0 0x7fd41406c290 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:40.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.680+0000 7fd42e833700 1 --2- 172.21.15.72:0/708478297 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd4280b23c0 0x7fd4281f6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:40.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.680+0000 7fd42e833700 1 --2- 172.21.15.72:0/708478297 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd4280d1710 0x7fd4280d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:40.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.680+0000 7fd42e833700 1 -- 172.21.15.72:0/708478297 >> 172.21.15.72:0/708478297 conn(0x7fd428066140 msgr2=0x7fd4280b4410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:40.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.680+0000 7fd42e833700 1 -- 172.21.15.72:0/708478297 shutdown_connections 2023-03-24T23:47:40.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.680+0000 7fd42e833700 1 -- 172.21.15.72:0/708478297 wait complete. 2023-03-24T23:47:40.693 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-24T23:47:40.821 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.817+0000 7f742e89e700 1 Processor -- start 2023-03-24T23:47:40.821 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.817+0000 7f742e89e700 1 -- start start 2023-03-24T23:47:40.821 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.818+0000 7f742e89e700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f74280cd450 0x7f74280cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:40.821 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.818+0000 7f742e89e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f74280cbc10 0x7f742815d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:40.822 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.818+0000 7f742e89e700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f742806a4a0 0x7f742815d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:40.822 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.818+0000 7f742e89e700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f742806beb0 con 0x7f74280cd450 2023-03-24T23:47:40.822 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.818+0000 7f742e89e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f742806b3c0 con 0x7f74280cbc10 2023-03-24T23:47:40.822 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.818+0000 7f742e89e700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74280cdd60 con 0x7f742806a4a0 2023-03-24T23:47:40.822 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.818+0000 7f7427fff700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f74280cd450 0x7f74280cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:40.822 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.819+0000 7f7427fff700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f74280cd450 0x7f74280cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54452/0 (socket says 172.21.15.72:54452) 2023-03-24T23:47:40.823 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.819+0000 7f7427fff700 1 -- 172.21.15.72:0/473696530 learned_addr learned my addr 172.21.15.72:0/473696530 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-03-24T23:47:40.823 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.819+0000 7f74277fe700 1 --2- 172.21.15.72:0/473696530 >> v2:172.21.15.146:3300/0 conn(0x7f74280cbc10 0x7f742815d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:40.823 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.819+0000 7f742ce3b700 1 --2- 172.21.15.72:0/473696530 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f742806a4a0 0x7f742815d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:40.823 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.819+0000 7f7427fff700 1 -- 172.21.15.72:0/473696530 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f742806a4a0 msgr2=0x7f742815d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T23:47:40.823 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.819+0000 7f7427fff700 1 --2- 172.21.15.72:0/473696530 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f742806a4a0 0x7f742815d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:40.824 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.819+0000 7f7427fff700 1 -- 172.21.15.72:0/473696530 >> v2:172.21.15.146:3300/0 conn(0x7f74280cbc10 msgr2=0x7f742815d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T23:47:40.824 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.819+0000 7f7427fff700 1 --2- 172.21.15.72:0/473696530 >> v2:172.21.15.146:3300/0 conn(0x7f74280cbc10 0x7f742815d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:40.824 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.819+0000 7f7427fff700 1 -- 172.21.15.72:0/473696530 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f742806b610 con 0x7f74280cd450 2023-03-24T23:47:40.824 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.819+0000 7f7427fff700 1 --2- 172.21.15.72:0/473696530 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74280cd450 0x7f74280cd820 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f74280ae610 tx=0x7f7418009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc446753eccfe4d server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:40.824 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.819+0000 7f7426ffd700 1 -- 172.21.15.72:0/473696530 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741800b870 con 0x7f74280cd450 2023-03-24T23:47:40.824 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.819+0000 7f7426ffd700 1 -- 172.21.15.72:0/473696530 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f741800b9d0 con 0x7f74280cd450 2023-03-24T23:47:40.824 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.819+0000 7f7426ffd700 1 -- 172.21.15.72:0/473696530 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741800bd60 con 0x7f74280cd450 2023-03-24T23:47:40.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.820+0000 7f742e89e700 1 -- 172.21.15.72:0/473696530 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74280cd450 msgr2=0x7f74280cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:40.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.820+0000 7f742e89e700 1 --2- 172.21.15.72:0/473696530 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74280cd450 0x7f74280cd820 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f74280ae610 tx=0x7f7418009bd0 comp rx=0 tx=0).stop 2023-03-24T23:47:40.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.820+0000 7f742e89e700 1 -- 172.21.15.72:0/473696530 shutdown_connections 2023-03-24T23:47:40.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.820+0000 7f742e89e700 1 --2- 172.21.15.72:0/473696530 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f742806a4a0 0x7f742815d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:40.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.820+0000 7f742e89e700 1 --2- 172.21.15.72:0/473696530 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74280cd450 0x7f74280cd820 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:40.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.820+0000 7f742e89e700 1 --2- 172.21.15.72:0/473696530 >> v2:172.21.15.146:3300/0 conn(0x7f74280cbc10 0x7f742815d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:40.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.820+0000 7f742e89e700 1 -- 172.21.15.72:0/473696530 >> 172.21.15.72:0/473696530 conn(0x7f74280ccd80 msgr2=0x7f74280cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:40.826 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.820+0000 7f742e89e700 1 -- 172.21.15.72:0/473696530 shutdown_connections 2023-03-24T23:47:40.826 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.820+0000 7f742e89e700 1 -- 172.21.15.72:0/473696530 wait complete. 2023-03-24T23:47:40.828 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.821+0000 7f742e89e700 1 Processor -- start 2023-03-24T23:47:40.828 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.821+0000 7f742e89e700 1 -- start start 2023-03-24T23:47:40.828 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.822+0000 7f742e89e700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f742806a4a0 0x7f7428150e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:40.828 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.822+0000 7f742e89e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74280cbc10 0x7f74281f0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:40.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.822+0000 7f742e89e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74280cd450 0x7f74280b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:40.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.822+0000 7f742e89e700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74280c8a70 con 0x7f74280cbc10 2023-03-24T23:47:40.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.822+0000 7f742e89e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74280c7c10 con 0x7f74280cd450 2023-03-24T23:47:40.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.822+0000 7f742e89e700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74280c7ef0 con 0x7f742806a4a0 2023-03-24T23:47:40.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.822+0000 7f74277fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74280cbc10 0x7f74281f0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:40.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.822+0000 7f74277fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74280cbc10 0x7f74281f0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54464/0 (socket says 172.21.15.72:54464) 2023-03-24T23:47:40.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.822+0000 7f74277fe700 1 -- 172.21.15.72:0/3569350995 learned_addr learned my addr 172.21.15.72:0/3569350995 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-03-24T23:47:40.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.823+0000 7f742ce3b700 1 --2- 172.21.15.72:0/3569350995 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74280cd450 0x7f74280b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:40.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.823+0000 7f74277fe700 1 -- 172.21.15.72:0/3569350995 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f742806a4a0 msgr2=0x7f7428150e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:40.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.823+0000 7f74277fe700 1 --2- 172.21.15.72:0/3569350995 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f742806a4a0 0x7f7428150e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:40.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.823+0000 7f74277fe700 1 -- 172.21.15.72:0/3569350995 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74280cd450 msgr2=0x7f74280b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:40.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.823+0000 7f74277fe700 1 --2- 172.21.15.72:0/3569350995 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74280cd450 0x7f74280b2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:40.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.823+0000 7f74277fe700 1 -- 172.21.15.72:0/3569350995 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7418009550 con 0x7f74280cbc10 2023-03-24T23:47:40.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.823+0000 7f74277fe700 1 --2- 172.21.15.72:0/3569350995 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74280cbc10 0x7f74281f0420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f741c0106b0 tx=0x7f741c010a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:40.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.823+0000 7f7424ff9700 1 -- 172.21.15.72:0/3569350995 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c011920 con 0x7f74280cbc10 2023-03-24T23:47:40.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.823+0000 7f7424ff9700 1 -- 172.21.15.72:0/3569350995 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f741c011a80 con 0x7f74280cbc10 2023-03-24T23:47:40.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.823+0000 7f742e89e700 1 -- 172.21.15.72:0/3569350995 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f742806bbd0 con 0x7f74280cbc10 2023-03-24T23:47:40.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.823+0000 7f7424ff9700 1 -- 172.21.15.72:0/3569350995 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c011e10 con 0x7f74280cbc10 2023-03-24T23:47:40.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.823+0000 7f742e89e700 1 -- 172.21.15.72:0/3569350995 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f742806b8f0 con 0x7f74280cbc10 2023-03-24T23:47:40.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.824+0000 7f742e89e700 1 -- 172.21.15.72:0/3569350995 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f742806d030 con 0x7f74280cbc10 2023-03-24T23:47:40.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.824+0000 7f7424ff9700 1 -- 172.21.15.72:0/3569350995 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f741c019440 con 0x7f74280cbc10 2023-03-24T23:47:40.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.824+0000 7f7424ff9700 1 --2- 172.21.15.72:0/3569350995 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f7414069de0 0x7f741406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:40.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.825+0000 7f7424ff9700 1 -- 172.21.15.72:0/3569350995 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f741c0899c0 con 0x7f74280cbc10 2023-03-24T23:47:40.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.825+0000 7f7427fff700 1 --2- 172.21.15.72:0/3569350995 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f7414069de0 0x7f741406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:40.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.825+0000 7f7427fff700 1 --2- 172.21.15.72:0/3569350995 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f7414069de0 0x7f741406c290 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f74280ca940 tx=0x7f7418010000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:40.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:40.829+0000 7f7424ff9700 1 -- 172.21.15.72:0/3569350995 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f741c053160 con 0x7f74280cbc10 2023-03-24T23:47:41.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.012+0000 7f742e89e700 1 -- 172.21.15.72:0/3569350995 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f742804e850 con 0x7f74280cbc10 2023-03-24T23:47:41.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.012+0000 7f7424ff9700 1 -- 172.21.15.72:0/3569350995 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7f741c01e020 con 0x7f74280cbc10 2023-03-24T23:47:41.014 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 8 2023-03-24T23:47:41.015 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.014+0000 7f742e89e700 1 -- 172.21.15.72:0/3569350995 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f7414069de0 msgr2=0x7f741406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:41.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.014+0000 7f742e89e700 1 --2- 172.21.15.72:0/3569350995 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f7414069de0 0x7f741406c290 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f74280ca940 tx=0x7f7418010000 comp rx=0 tx=0).stop 2023-03-24T23:47:41.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.014+0000 7f742e89e700 1 -- 172.21.15.72:0/3569350995 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74280cbc10 msgr2=0x7f74281f0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:41.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.014+0000 7f742e89e700 1 --2- 172.21.15.72:0/3569350995 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74280cbc10 0x7f74281f0420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f741c0106b0 tx=0x7f741c010a70 comp rx=0 tx=0).stop 2023-03-24T23:47:41.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.014+0000 7f742e89e700 1 -- 172.21.15.72:0/3569350995 shutdown_connections 2023-03-24T23:47:41.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.014+0000 7f742e89e700 1 --2- 172.21.15.72:0/3569350995 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7f7414069de0 0x7f741406c290 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:41.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.014+0000 7f742e89e700 1 --2- 172.21.15.72:0/3569350995 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74280cd450 0x7f74280b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:41.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.014+0000 7f742e89e700 1 --2- 172.21.15.72:0/3569350995 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f742806a4a0 0x7f7428150e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:41.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.014+0000 7f742e89e700 1 --2- 172.21.15.72:0/3569350995 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74280cbc10 0x7f74281f0420 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:41.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.014+0000 7f742e89e700 1 -- 172.21.15.72:0/3569350995 >> 172.21.15.72:0/3569350995 conn(0x7f74280ccd80 msgr2=0x7f74280caf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:41.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.014+0000 7f742e89e700 1 -- 172.21.15.72:0/3569350995 shutdown_connections 2023-03-24T23:47:41.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.014+0000 7f742e89e700 1 -- 172.21.15.72:0/3569350995 wait complete. 2023-03-24T23:47:41.028 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-03-24T23:47:41.149 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.148+0000 7fd0216ca700 1 Processor -- start 2023-03-24T23:47:41.149 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.148+0000 7fd0216ca700 1 -- start start 2023-03-24T23:47:41.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.149+0000 7fd0216ca700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd01c06c1b0 0x7fd01c06a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:41.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.149+0000 7fd0216ca700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd01c071a80 0x7fd01c06ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:41.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.149+0000 7fd0216ca700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd01c06de10 con 0x7fd01c15cbd0 2023-03-24T23:47:41.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.149+0000 7fd0216ca700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd01c06e040 con 0x7fd01c071a80 2023-03-24T23:47:41.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.149+0000 7fd0216ca700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd01c15cfa0 con 0x7fd01c06c1b0 2023-03-24T23:47:41.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.149+0000 7fd013fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd01c06c1b0 0x7fd01c06a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:41.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.149+0000 7fd013fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd01c06c1b0 0x7fd01c06a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.72:47516/0 (socket says 172.21.15.72:47516) 2023-03-24T23:47:41.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.149+0000 7fd013fff700 1 -- 172.21.15.72:0/1286907462 learned_addr learned my addr 172.21.15.72:0/1286907462 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-03-24T23:47:41.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.149+0000 7fd01b7fe700 1 --2- 172.21.15.72:0/1286907462 >> v2:172.21.15.146:3300/0 conn(0x7fd01c071a80 0x7fd01c06ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:41.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.149+0000 7fd01a7fc700 1 -- 172.21.15.72:0/1286907462 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3686881088 0 0) 0x7fd01c06de10 con 0x7fd01c15cbd0 2023-03-24T23:47:41.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.149+0000 7fd01a7fc700 1 -- 172.21.15.72:0/1286907462 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0080035b0 con 0x7fd01c15cbd0 2023-03-24T23:47:41.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.150+0000 7fd013fff700 1 -- 172.21.15.72:0/1286907462 >> v2:172.21.15.146:3300/0 conn(0x7fd01c071a80 msgr2=0x7fd01c06ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T23:47:41.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.150+0000 7fd013fff700 1 --2- 172.21.15.72:0/1286907462 >> v2:172.21.15.146:3300/0 conn(0x7fd01c071a80 0x7fd01c06ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:41.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.150+0000 7fd013fff700 1 -- 172.21.15.72:0/1286907462 >> v1:172.21.15.146:6789/0 conn(0x7fd01c15cbd0 legacy=0x7fd01c06baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:41.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.150+0000 7fd013fff700 1 -- 172.21.15.72:0/1286907462 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd01c0cde00 con 0x7fd01c06c1b0 2023-03-24T23:47:41.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.150+0000 7fd013fff700 1 --2- 172.21.15.72:0/1286907462 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd01c06c1b0 0x7fd01c06a740 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd0040098f0 tx=0x7fd004009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99daae1ee0529b52 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:41.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.150+0000 7fd01a7fc700 1 -- 172.21.15.72:0/1286907462 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd004011830 con 0x7fd01c06c1b0 2023-03-24T23:47:41.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.150+0000 7fd01a7fc700 1 -- 172.21.15.72:0/1286907462 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd004011990 con 0x7fd01c06c1b0 2023-03-24T23:47:41.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.150+0000 7fd01a7fc700 1 -- 172.21.15.72:0/1286907462 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd004011d20 con 0x7fd01c06c1b0 2023-03-24T23:47:41.156 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.150+0000 7fd0216ca700 1 -- 172.21.15.72:0/1286907462 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd01c06c1b0 msgr2=0x7fd01c06a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:41.156 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.150+0000 7fd0216ca700 1 --2- 172.21.15.72:0/1286907462 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd01c06c1b0 0x7fd01c06a740 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd0040098f0 tx=0x7fd004009c00 comp rx=0 tx=0).stop 2023-03-24T23:47:41.157 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.150+0000 7fd0216ca700 1 -- 172.21.15.72:0/1286907462 shutdown_connections 2023-03-24T23:47:41.157 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.150+0000 7fd0216ca700 1 --2- 172.21.15.72:0/1286907462 >> v2:172.21.15.146:3300/0 conn(0x7fd01c071a80 0x7fd01c06ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:41.157 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.150+0000 7fd0216ca700 1 --2- 172.21.15.72:0/1286907462 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd01c06c1b0 0x7fd01c06a740 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:41.157 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.150+0000 7fd0216ca700 1 -- 172.21.15.72:0/1286907462 >> 172.21.15.72:0/1286907462 conn(0x7fd01c0cc980 msgr2=0x7fd01c0ccd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:41.157 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.151+0000 7fd0216ca700 1 -- 172.21.15.72:0/1286907462 shutdown_connections 2023-03-24T23:47:41.157 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.151+0000 7fd0216ca700 1 -- 172.21.15.72:0/1286907462 wait complete. 2023-03-24T23:47:41.157 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.152+0000 7fd0216ca700 1 Processor -- start 2023-03-24T23:47:41.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.152+0000 7fd0216ca700 1 -- start start 2023-03-24T23:47:41.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.152+0000 7fd0216ca700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd01c06c1b0 0x7fd01c156450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:41.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.152+0000 7fd0216ca700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd01c071a80 0x7fd01c15d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:41.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.152+0000 7fd0216ca700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd01c15cbd0 0x7fd01c15d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:41.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.152+0000 7fd0216ca700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01c156990 con 0x7fd01c15cbd0 2023-03-24T23:47:41.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.152+0000 7fd0216ca700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01c0c7fe0 con 0x7fd01c071a80 2023-03-24T23:47:41.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.152+0000 7fd0216ca700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd01c0c7d00 con 0x7fd01c06c1b0 2023-03-24T23:47:41.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.152+0000 7fd01b7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd01c15cbd0 0x7fd01c15d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:41.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.153+0000 7fd01b7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd01c15cbd0 0x7fd01c15d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54470/0 (socket says 172.21.15.72:54470) 2023-03-24T23:47:41.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.153+0000 7fd01b7fe700 1 -- 172.21.15.72:0/4136205012 learned_addr learned my addr 172.21.15.72:0/4136205012 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-03-24T23:47:41.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.153+0000 7fd013fff700 1 --2- 172.21.15.72:0/4136205012 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd01c071a80 0x7fd01c15d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:41.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.153+0000 7fd01affd700 1 --2- 172.21.15.72:0/4136205012 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd01c06c1b0 0x7fd01c156450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:41.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.153+0000 7fd01b7fe700 1 -- 172.21.15.72:0/4136205012 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd01c06c1b0 msgr2=0x7fd01c156450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:41.160 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.153+0000 7fd01b7fe700 1 --2- 172.21.15.72:0/4136205012 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd01c06c1b0 0x7fd01c156450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:41.160 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.153+0000 7fd01b7fe700 1 -- 172.21.15.72:0/4136205012 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd01c071a80 msgr2=0x7fd01c15d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:41.160 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.153+0000 7fd01b7fe700 1 --2- 172.21.15.72:0/4136205012 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd01c071a80 0x7fd01c15d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:41.160 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.153+0000 7fd01b7fe700 1 -- 172.21.15.72:0/4136205012 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd004009550 con 0x7fd01c15cbd0 2023-03-24T23:47:41.160 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.153+0000 7fd01affd700 1 --2- 172.21.15.72:0/4136205012 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd01c06c1b0 0x7fd01c156450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-24T23:47:41.160 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.153+0000 7fd01b7fe700 1 --2- 172.21.15.72:0/4136205012 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd01c15cbd0 0x7fd01c15d810 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd01400efc0 tx=0x7fd01400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:41.160 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.153+0000 7fd0137fe700 1 -- 172.21.15.72:0/4136205012 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd014009ea0 con 0x7fd01c15cbd0 2023-03-24T23:47:41.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.153+0000 7fd0137fe700 1 -- 172.21.15.72:0/4136205012 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd01400f040 con 0x7fd01c15cbd0 2023-03-24T23:47:41.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.153+0000 7fd0216ca700 1 -- 172.21.15.72:0/4136205012 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd01c06e5c0 con 0x7fd01c15cbd0 2023-03-24T23:47:41.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.153+0000 7fd0137fe700 1 -- 172.21.15.72:0/4136205012 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd014014480 con 0x7fd01c15cbd0 2023-03-24T23:47:41.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.153+0000 7fd0216ca700 1 -- 172.21.15.72:0/4136205012 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd01c06e2e0 con 0x7fd01c15cbd0 2023-03-24T23:47:41.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.154+0000 7fd0216ca700 1 -- 172.21.15.72:0/4136205012 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd01c06e8a0 con 0x7fd01c15cbd0 2023-03-24T23:47:41.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.155+0000 7fd0137fe700 1 -- 172.21.15.72:0/4136205012 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fd0140145e0 con 0x7fd01c15cbd0 2023-03-24T23:47:41.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.155+0000 7fd0137fe700 1 --2- 172.21.15.72:0/4136205012 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7fcffc069de0 0x7fcffc06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:41.162 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.155+0000 7fd0137fe700 1 -- 172.21.15.72:0/4136205012 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fd014088590 con 0x7fd01c15cbd0 2023-03-24T23:47:41.162 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.155+0000 7fd01affd700 1 --2- 172.21.15.72:0/4136205012 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7fcffc069de0 0x7fcffc06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:41.162 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.155+0000 7fd01affd700 1 --2- 172.21.15.72:0/4136205012 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7fcffc069de0 0x7fcffc06c290 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd00c0011e0 tx=0x7fd00c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:41.162 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.159+0000 7fd0137fe700 1 -- 172.21.15.72:0/4136205012 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fd014054100 con 0x7fd01c15cbd0 2023-03-24T23:47:41.349 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.347+0000 7fd0216ca700 1 -- 172.21.15.72:0/4136205012 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd01c0b5e80 con 0x7fd01c15cbd0 2023-03-24T23:47:41.349 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.348+0000 7fd0137fe700 1 -- 172.21.15.72:0/4136205012 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) v1 ==== 84+0+126 (secure 0 0 0) 0x7fd014018090 con 0x7fd01c15cbd0 2023-03-24T23:47:41.349 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-24T23:47:41.350 INFO:teuthology.orchestra.run.smithi072.stdout:[ 2023-03-24T23:47:41.350 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-03-24T23:47:41.350 INFO:teuthology.orchestra.run.smithi072.stdout: "poolnum": 1, 2023-03-24T23:47:41.350 INFO:teuthology.orchestra.run.smithi072.stdout: "poolname": ".mgr" 2023-03-24T23:47:41.350 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-03-24T23:47:41.350 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-03-24T23:47:41.350 INFO:teuthology.orchestra.run.smithi072.stdout: "poolnum": 2, 2023-03-24T23:47:41.351 INFO:teuthology.orchestra.run.smithi072.stdout: "poolname": "rbd" 2023-03-24T23:47:41.351 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-03-24T23:47:41.351 INFO:teuthology.orchestra.run.smithi072.stdout:] 2023-03-24T23:47:41.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.350+0000 7fd0216ca700 1 -- 172.21.15.72:0/4136205012 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7fcffc069de0 msgr2=0x7fcffc06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:41.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.350+0000 7fd0216ca700 1 --2- 172.21.15.72:0/4136205012 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7fcffc069de0 0x7fcffc06c290 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd00c0011e0 tx=0x7fd00c009040 comp rx=0 tx=0).stop 2023-03-24T23:47:41.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.350+0000 7fd0216ca700 1 -- 172.21.15.72:0/4136205012 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd01c15cbd0 msgr2=0x7fd01c15d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:41.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.350+0000 7fd0216ca700 1 --2- 172.21.15.72:0/4136205012 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd01c15cbd0 0x7fd01c15d810 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd01400efc0 tx=0x7fd01400c3a0 comp rx=0 tx=0).stop 2023-03-24T23:47:41.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.350+0000 7fd0216ca700 1 -- 172.21.15.72:0/4136205012 shutdown_connections 2023-03-24T23:47:41.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.350+0000 7fd0216ca700 1 --2- 172.21.15.72:0/4136205012 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd01c15cbd0 0x7fd01c15d810 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:41.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.350+0000 7fd0216ca700 1 --2- 172.21.15.72:0/4136205012 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd01c06c1b0 0x7fd01c156450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:41.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.350+0000 7fd0216ca700 1 --2- 172.21.15.72:0/4136205012 >> [v2:172.21.15.72:6832/114637,v1:172.21.15.72:6833/114637] conn(0x7fcffc069de0 0x7fcffc06c290 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:41.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.350+0000 7fd0216ca700 1 --2- 172.21.15.72:0/4136205012 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd01c071a80 0x7fd01c15d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:41.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.350+0000 7fd0216ca700 1 -- 172.21.15.72:0/4136205012 >> 172.21.15.72:0/4136205012 conn(0x7fd01c0cc980 msgr2=0x7fd01c0ca690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T23:47:41.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.350+0000 7fd0216ca700 1 -- 172.21.15.72:0/4136205012 shutdown_connections 2023-03-24T23:47:41.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.350+0000 7fd0216ca700 1 -- 172.21.15.72:0/4136205012 wait complete. 2023-03-24T23:47:41.363 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-03-24T23:47:41.363 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-03-24T23:47:41.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.486+0000 7fd35c093700 1 Processor -- start 2023-03-24T23:47:41.488 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.486+0000 7fd35c093700 1 -- start start 2023-03-24T23:47:41.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.487+0000 7fd35c093700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd35406e090 0x7fd35406e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:41.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.487+0000 7fd35c093700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3540cfbc0 0x7fd35406b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:41.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.487+0000 7fd35c093700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd35406e9a0 con 0x7fd35406cf50 2023-03-24T23:47:41.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.487+0000 7fd35c093700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd35406cb30 con 0x7fd35406e090 2023-03-24T23:47:41.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.487+0000 7fd35c093700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd35406beb0 con 0x7fd3540cfbc0 2023-03-24T23:47:41.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.487+0000 7fd35a630700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fd35406cf50 0x7fd3540adaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:50034/0 (socket says 172.21.15.72:50034) 2023-03-24T23:47:41.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.487+0000 7fd35a630700 1 -- 172.21.15.72:0/2903456886 learned_addr learned my addr 172.21.15.72:0/2903456886 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-03-24T23:47:41.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.488+0000 7fd359e2f700 1 --2- 172.21.15.72:0/2903456886 >> v2:172.21.15.146:3300/0 conn(0x7fd35406e090 0x7fd35406e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:41.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.488+0000 7fd35962e700 1 --2- 172.21.15.72:0/2903456886 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3540cfbc0 0x7fd35406b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:41.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.488+0000 7fd358e2d700 1 -- 172.21.15.72:0/2903456886 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3455678890 0 0) 0x7fd35406e9a0 con 0x7fd35406cf50 2023-03-24T23:47:41.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.488+0000 7fd358e2d700 1 -- 172.21.15.72:0/2903456886 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3400035b0 con 0x7fd35406cf50 2023-03-24T23:47:41.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.488+0000 7fd358e2d700 1 -- 172.21.15.72:0/2903456886 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2609268832 0 0) 0x7fd3400035b0 con 0x7fd35406cf50 2023-03-24T23:47:41.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.488+0000 7fd358e2d700 1 -- 172.21.15.72:0/2903456886 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3540cfbc0 msgr2=0x7fd35406b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T23:47:41.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.488+0000 7fd358e2d700 1 --2- 172.21.15.72:0/2903456886 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3540cfbc0 0x7fd35406b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:41.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.488+0000 7fd358e2d700 1 -- 172.21.15.72:0/2903456886 >> v2:172.21.15.146:3300/0 conn(0x7fd35406e090 msgr2=0x7fd35406e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T23:47:41.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.488+0000 7fd358e2d700 1 --2- 172.21.15.72:0/2903456886 >> v2:172.21.15.146:3300/0 conn(0x7fd35406e090 0x7fd35406e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:41.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.488+0000 7fd358e2d700 1 -- 172.21.15.72:0/2903456886 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd35406c190 con 0x7fd35406cf50 2023-03-24T23:47:41.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.488+0000 7fd358e2d700 1 -- 172.21.15.72:0/2903456886 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 293402998 0 0) 0x7fd34c0019f0 con 0x7fd35406cf50 2023-03-24T23:47:41.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.488+0000 7fd358e2d700 1 -- 172.21.15.72:0/2903456886 >> v1:172.21.15.72:6789/0 conn(0x7fd35406cf50 legacy=0x7fd3540adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T23:47:41.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.488+0000 7fd358e2d700 1 --2- 172.21.15.72:0/2903456886 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd340003a00 0x7fd340003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:41.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.488+0000 7fd358e2d700 1 --2- 172.21.15.72:0/2903456886 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3540cfbc0 0x7fd340004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:41.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.488+0000 7fd358e2d700 1 --2- 172.21.15.72:0/2903456886 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd340004d90 0x7fd340005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T23:47:41.494 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.488+0000 7fd358e2d700 1 -- 172.21.15.72:0/2903456886 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35406cb30 con 0x7fd340003a00 2023-03-24T23:47:41.494 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.488+0000 7fd358e2d700 1 -- 172.21.15.72:0/2903456886 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35406beb0 con 0x7fd340004d90 2023-03-24T23:47:41.494 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.488+0000 7fd358e2d700 1 -- 172.21.15.72:0/2903456886 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd34000ba10 con 0x7fd3540cfbc0 2023-03-24T23:47:41.494 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.488+0000 7fd359e2f700 1 --2- 172.21.15.72:0/2903456886 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd340003a00 0x7fd340003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:41.494 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.488+0000 7fd35962e700 1 --2- 172.21.15.72:0/2903456886 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3540cfbc0 0x7fd340004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:41.494 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.488+0000 7fd35a630700 1 --2- 172.21.15.72:0/2903456886 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd340004d90 0x7fd340005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T23:47:41.494 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.488+0000 7fd359e2f700 1 -- 172.21.15.72:0/2903456886 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3540cfbc0 msgr2=0x7fd340004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T23:47:41.495 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.488+0000 7fd359e2f700 1 --2- 172.21.15.72:0/2903456886 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3540cfbc0 0x7fd340004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:41.495 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.488+0000 7fd359e2f700 1 -- 172.21.15.72:0/2903456886 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd340004d90 msgr2=0x7fd340005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T23:47:41.495 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.488+0000 7fd359e2f700 1 --2- 172.21.15.72:0/2903456886 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd340004d90 0x7fd340005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T23:47:41.495 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.488+0000 7fd359e2f700 1 -- 172.21.15.72:0/2903456886 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3400035b0 con 0x7fd340003a00 2023-03-24T23:47:41.495 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.488+0000 7fd359e2f700 1 --2- 172.21.15.72:0/2903456886 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd340003a00 0x7fd340003e10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd350009b40 tx=0x7fd340005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a1c0e6ee79bcc8 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T23:47:41.495 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.489+0000 7fd358e2d700 1 -- 172.21.15.72:0/2903456886 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd350005a20 con 0x7fd340003a00 2023-03-24T23:47:41.495 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-24T23:47:41.489+0000 7fd358e2d700 1 -- 172.21.15.72:0/2903456886 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd35