2023-03-27T09:44:44.497 INFO:root:teuthology version: 0.0.1.dev1+ge372916 2023-03-27T09:44:44.498 DEBUG:teuthology.run:Teuthology command: teuthology --name rishabh-2023-03-27_06:39:30-fs-wip-rishabh-cephman-6-testing-default-smithi --owner scheduled_rishabh@teuthology --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/snaps}} --verbose --archive /home/teuthworker/archive/rishabh-2023-03-27_06:39:30-fs-wip-rishabh-cephman-6-testing-default-smithi/7221293 -- /home/teuthworker/archive/rishabh-2023-03-27_06:39:30-fs-wip-rishabh-cephman-6-testing-default-smithi/7221293/orig.config.yaml 2023-03-27T09:44:44.563 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2023-03-27T09:44:44.645 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-03-27_06:39:30-fs-wip-rishabh-cephman-6-testing-default-smithi/7221293 branch: wip-rishabh-cephman-6 description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/snaps}} email: null first_in_suite: false job_id: '7221293' kernel: client: branch: testing flavor: default kdb: true sha1: 1776e8f494e94fd3414d7679e5782ec22e83c95f last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-03-27_06:39:30-fs-wip-rishabh-cephman-6-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-rishabh-cephman-6 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds oft prefetch dirfrags: false mds op complaint time: 180 mds verify scatter: true mds_dir_prefetch: true mds_sleep_rank_change: 5000000.0 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - overall HEALTH_ - \(MON_DOWN\) sha1: b70fdf973c2ffa23c0d8aa6918519da05d922302 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: b70fdf973c2ffa23c0d8aa6918519da05d922302 kclient: syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: CephManager-in-CephFSTestCase sha1: 342392241eddd9827618f3dbb8134df7441718a8 owner: scheduled_rishabh@teuthology priority: 200 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - mds.e - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - mds.f - osd.4 - osd.5 - osd.6 - osd.7 sha1: b70fdf973c2ffa23c0d8aa6918519da05d922302 sleep_before_teardown: 0 suite: fs suite_branch: CephManager-in-CephFSTestCase suite_path: /home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa suite_relpath: qa suite_repo: https://github.com/rishabh-d-dave/ceph suite_sha1: 342392241eddd9827618f3dbb8134df7441718a8 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi023.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi112.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - kclient: null - mon_thrash: check_mds_failover: true revive_delay: 20 thrash_delay: 10 - workunit: clients: all: - fs/snaps timeout: 6h teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: e3729161eecf8e6d09a4069acbb883cdcd5e9f50 timestamp: 2023-03-27_06:39:30 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.4538 2023-03-27T09:44:44.645 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa; will attempt to use it 2023-03-27T09:44:44.646 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa/tasks 2023-03-27T09:44:44.646 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-03-27T09:44:44.648 INFO:teuthology.task.internal:Checking packages... 2023-03-27T09:44:44.689 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'b70fdf973c2ffa23c0d8aa6918519da05d922302' 2023-03-27T09:44:44.690 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-03-27T09:44:44.690 INFO:teuthology.packaging:ref: None 2023-03-27T09:44:44.690 INFO:teuthology.packaging:tag: None 2023-03-27T09:44:44.691 INFO:teuthology.packaging:branch: wip-rishabh-cephman-6 2023-03-27T09:44:44.691 INFO:teuthology.packaging:sha1: b70fdf973c2ffa23c0d8aa6918519da05d922302 2023-03-27T09:44:44.691 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-rishabh-cephman-6 2023-03-27T09:44:44.830 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-2533.gb70fdf97 2023-03-27T09:44:44.831 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-03-27T09:44:44.840 INFO:teuthology.task.internal:no buildpackages task found 2023-03-27T09:44:44.841 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-03-27T09:44:44.874 INFO:teuthology.task.internal:Saving configuration 2023-03-27T09:44:44.898 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-03-27T09:44:44.905 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-03-27T09:44:44.933 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi023.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-03-27_06:39:30-fs-wip-rishabh-cephman-6-testing-default-smithi/7221293', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-03-27 09:38:51.727163', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-03-27T09:44:44.960 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi112.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-03-27_06:39:30-fs-wip-rishabh-cephman-6-testing-default-smithi/7221293', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-03-27 09:38:51.728859', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-03-27T09:44:44.960 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-03-27T09:44:44.971 INFO:teuthology.task.internal:roles: ubuntu@smithi023.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2023-03-27T09:44:44.971 INFO:teuthology.task.internal:roles: ubuntu@smithi112.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2023-03-27T09:44:44.972 INFO:teuthology.run_tasks:Running task console_log... 2023-03-27T09:44:45.063 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6e1b6e44c0>, signals=[15]) 2023-03-27T09:44:45.065 INFO:teuthology.run_tasks:Running task internal.connect... 2023-03-27T09:44:45.071 INFO:teuthology.task.internal:Opening connections... 2023-03-27T09:44:45.072 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi023.front.sepia.ceph.com 2023-03-27T09:44:45.074 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-27T09:44:45.143 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi112.front.sepia.ceph.com 2023-03-27T09:44:45.145 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-27T09:44:45.215 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-03-27T09:44:45.222 DEBUG:teuthology.orchestra.run.smithi023:> uname -m 2023-03-27T09:44:45.245 INFO:teuthology.orchestra.run.smithi023.stdout:x86_64 2023-03-27T09:44:45.246 DEBUG:teuthology.orchestra.run.smithi023:> cat /etc/os-release 2023-03-27T09:44:45.305 INFO:teuthology.orchestra.run.smithi023.stdout:NAME="Red Hat Enterprise Linux" 2023-03-27T09:44:45.305 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION="8.6 (Ootpa)" 2023-03-27T09:44:45.305 INFO:teuthology.orchestra.run.smithi023.stdout:ID="rhel" 2023-03-27T09:44:45.305 INFO:teuthology.orchestra.run.smithi023.stdout:ID_LIKE="fedora" 2023-03-27T09:44:45.306 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION_ID="8.6" 2023-03-27T09:44:45.306 INFO:teuthology.orchestra.run.smithi023.stdout:PLATFORM_ID="platform:el8" 2023-03-27T09:44:45.306 INFO:teuthology.orchestra.run.smithi023.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-03-27T09:44:45.306 INFO:teuthology.orchestra.run.smithi023.stdout:ANSI_COLOR="0;31" 2023-03-27T09:44:45.306 INFO:teuthology.orchestra.run.smithi023.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-03-27T09:44:45.306 INFO:teuthology.orchestra.run.smithi023.stdout:HOME_URL="https://www.redhat.com/" 2023-03-27T09:44:45.306 INFO:teuthology.orchestra.run.smithi023.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-03-27T09:44:45.307 INFO:teuthology.orchestra.run.smithi023.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-03-27T09:44:45.307 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T09:44:45.307 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-03-27T09:44:45.307 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-03-27T09:44:45.307 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-03-27T09:44:45.307 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-03-27T09:44:45.309 INFO:teuthology.lock.ops:Updating smithi023.front.sepia.ceph.com on lock server 2023-03-27T09:44:45.345 DEBUG:teuthology.orchestra.run.smithi112:> uname -m 2023-03-27T09:44:45.365 INFO:teuthology.orchestra.run.smithi112.stdout:x86_64 2023-03-27T09:44:45.366 DEBUG:teuthology.orchestra.run.smithi112:> cat /etc/os-release 2023-03-27T09:44:45.424 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="Red Hat Enterprise Linux" 2023-03-27T09:44:45.425 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="8.6 (Ootpa)" 2023-03-27T09:44:45.425 INFO:teuthology.orchestra.run.smithi112.stdout:ID="rhel" 2023-03-27T09:44:45.425 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE="fedora" 2023-03-27T09:44:45.425 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="8.6" 2023-03-27T09:44:45.425 INFO:teuthology.orchestra.run.smithi112.stdout:PLATFORM_ID="platform:el8" 2023-03-27T09:44:45.426 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-03-27T09:44:45.426 INFO:teuthology.orchestra.run.smithi112.stdout:ANSI_COLOR="0;31" 2023-03-27T09:44:45.426 INFO:teuthology.orchestra.run.smithi112.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-03-27T09:44:45.426 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://www.redhat.com/" 2023-03-27T09:44:45.426 INFO:teuthology.orchestra.run.smithi112.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-03-27T09:44:45.426 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-03-27T09:44:45.426 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-03-27T09:44:45.426 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-03-27T09:44:45.427 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-03-27T09:44:45.427 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-03-27T09:44:45.427 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-03-27T09:44:45.427 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2023-03-27T09:44:45.453 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-03-27T09:44:45.464 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-03-27T09:44:45.475 INFO:teuthology.task.internal:Checking for old test directory... 2023-03-27T09:44:45.475 DEBUG:teuthology.orchestra.run.smithi023:> test '!' -e /home/ubuntu/cephtest 2023-03-27T09:44:45.479 DEBUG:teuthology.orchestra.run.smithi112:> test '!' -e /home/ubuntu/cephtest 2023-03-27T09:44:45.498 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-03-27T09:44:45.510 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-03-27T09:44:45.510 DEBUG:teuthology.orchestra.run.smithi023:> test -z $(ls -A /var/lib/ceph) 2023-03-27T09:44:45.538 DEBUG:teuthology.orchestra.run.smithi112:> test -z $(ls -A /var/lib/ceph) 2023-03-27T09:44:45.568 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-03-27T09:44:45.632 INFO:teuthology.run_tasks:Running task kernel... 2023-03-27T09:44:45.665 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '1776e8f494e94fd3414d7679e5782ec22e83c95f'} 2023-03-27T09:44:45.666 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2023-03-27T09:44:45.667 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2023-03-27T09:44:45.668 DEBUG:teuthology.orchestra.run.smithi023:> test -f /run/.containerenv -o -f /.dockerenv 2023-03-27T09:44:45.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-27T09:44:45.690 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-03-27T09:44:45.875 DEBUG:teuthology.task.kernel:sha1 for client.0 is 1776e8f494e94fd3414d7679e5782ec22e83c95f 2023-03-27T09:44:45.876 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "1776e8f494e94fd3414d7679e5782ec22e83c95f"... 2023-03-27T09:44:45.876 DEBUG:teuthology.orchestra.run.smithi023:> uname -r 2023-03-27T09:44:45.897 INFO:teuthology.orchestra.run.smithi023.stdout:4.18.0-372.9.1.el8.x86_64 2023-03-27T09:44:45.898 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs 1776e8f494e94fd3414d7679e5782ec22e83c95f 2023-03-27T09:44:45.899 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-03-27T09:44:45.900 DEBUG:teuthology.orchestra.run.smithi023:> sudo yum upgrade -y linux-firmware 2023-03-27T09:44:47.047 INFO:teuthology.orchestra.run.smithi023.stdout:Updating Subscription Management repositories. 2023-03-27T09:44:47.048 INFO:teuthology.orchestra.run.smithi023.stdout:Unable to read consumer identity 2023-03-27T09:44:48.460 INFO:teuthology.orchestra.run.smithi023.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2023-03-27T09:44:49.444 INFO:teuthology.orchestra.run.smithi023.stdout:Extra Packages for Enterprise Linux 14 MB/s | 14 MB 00:00 2023-03-27T09:44:52.662 INFO:teuthology.orchestra.run.smithi023.stdout:lab-extras 525 kB/s | 24 kB 00:00 2023-03-27T09:44:53.378 INFO:teuthology.orchestra.run.smithi023.stdout:Dependencies resolved. 2023-03-27T09:44:53.378 INFO:teuthology.orchestra.run.smithi023.stdout:Nothing to do. 2023-03-27T09:44:53.379 INFO:teuthology.orchestra.run.smithi023.stdout:Complete! 2023-03-27T09:44:53.517 INFO:teuthology.task.kernel:Downloading kernel 1776e8f494e94fd3414d7679e5782ec22e83c95f on client.0... 2023-03-27T09:44:53.519 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1776e8f494e94fd3414d7679e5782ec22e83c95f 2023-03-27T09:44:53.667 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/1776e8f494e94fd3414d7679e5782ec22e83c95f/centos/8/flavors/default/x86_64/ 2023-03-27T09:44:53.667 DEBUG:teuthology.orchestra.run.smithi023:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.3.0_rc3_g1776e8f494e9-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/1776e8f494e94fd3414d7679e5782ec22e83c95f/centos/8/flavors/default/x86_64/ --input-file=- 2023-03-27T09:44:53.679 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi023.front.sepia.ceph.com to complete... 2023-03-27T09:45:06.443 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27 09:45:06 URL:https://2.chacra.ceph.com/r/kernel/testing/1776e8f494e94fd3414d7679e5782ec22e83c95f/centos/8/flavors/default/x86_64/kernel-6.3.0_rc3_g1776e8f494e9-1.x86_64.rpm [377992976/377992976] -> "/tmp/kernel.x86_64.rpm" [1] 2023-03-27T09:45:06.443 INFO:teuthology.orchestra.run.smithi023.stderr:FINISHED --2023-03-27 09:45:06-- 2023-03-27T09:45:06.444 INFO:teuthology.orchestra.run.smithi023.stderr:Total wall clock time: 13s 2023-03-27T09:45:06.444 INFO:teuthology.orchestra.run.smithi023.stderr:Downloaded: 1 files, 360M in 13s (28.8 MB/s) 2023-03-27T09:45:06.446 INFO:teuthology.task.kernel:Installing kernel 1776e8f494e94fd3414d7679e5782ec22e83c95f on client.0... 2023-03-27T09:45:06.447 DEBUG:teuthology.orchestra.run.smithi023:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-03-27T09:45:08.080 INFO:teuthology.orchestra.run.smithi023.stdout:Verifying... ######################################## 2023-03-27T09:45:08.131 INFO:teuthology.orchestra.run.smithi023.stdout:Preparing... ######################################## 2023-03-27T09:45:08.220 INFO:teuthology.orchestra.run.smithi023.stdout:Updating / installing... 2023-03-27T09:45:48.367 INFO:teuthology.orchestra.run.smithi023.stdout:kernel-6.3.0_rc3_g1776e8f494e9-1 ######################################## 2023-03-27T09:46:13.246 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi023.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-03-27T09:46:13.247 DEBUG:teuthology.orchestra.run.smithi023:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-03-27T09:46:13.302 INFO:teuthology.orchestra.run.smithi023.stdout:/boot/System.map-6.3.0-rc3-g1776e8f494e9 2023-03-27T09:46:13.303 INFO:teuthology.orchestra.run.smithi023.stdout:/boot/config-6.3.0-rc3-g1776e8f494e9 2023-03-27T09:46:13.303 INFO:teuthology.orchestra.run.smithi023.stdout:/boot/vmlinuz-6.3.0-rc3-g1776e8f494e9 2023-03-27T09:46:13.303 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9 2023-03-27T09:46:13.303 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel 2023-03-27T09:46:13.303 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch 2023-03-27T09:46:13.304 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86 2023-03-27T09:46:13.304 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto 2023-03-27T09:46:13.304 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-03-27T09:46:13.304 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-03-27T09:46:13.304 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-03-27T09:46:13.304 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-03-27T09:46:13.305 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-03-27T09:46:13.305 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-03-27T09:46:13.305 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-03-27T09:46:13.305 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/crc32c-intel.ko 2023-03-27T09:46:13.305 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-03-27T09:46:13.305 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-03-27T09:46:13.305 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-03-27T09:46:13.306 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-03-27T09:46:13.306 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/serpent-avx2.ko 2023-03-27T09:46:13.306 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-03-27T09:46:13.306 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-03-27T09:46:13.306 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-03-27T09:46:13.306 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-03-27T09:46:13.306 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-03-27T09:46:13.307 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-03-27T09:46:13.307 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-03-27T09:46:13.307 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/kernel 2023-03-27T09:46:13.307 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/kernel/cpu 2023-03-27T09:46:13.307 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/kernel/cpu/mce 2023-03-27T09:46:13.308 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-03-27T09:46:13.308 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/kvm 2023-03-27T09:46:13.308 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/kvm/kvm-amd.ko 2023-03-27T09:46:13.308 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/kvm/kvm-intel.ko 2023-03-27T09:46:13.308 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/kvm/kvm.ko 2023-03-27T09:46:13.308 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto 2023-03-27T09:46:13.308 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/ansi_cprng.ko 2023-03-27T09:46:13.309 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/anubis.ko 2023-03-27T09:46:13.309 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/arc4.ko 2023-03-27T09:46:13.309 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/async_tx 2023-03-27T09:46:13.309 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/async_tx/async_memcpy.ko 2023-03-27T09:46:13.309 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/async_tx/async_pq.ko 2023-03-27T09:46:13.309 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/async_tx/async_raid6_recov.ko 2023-03-27T09:46:13.310 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/async_tx/async_tx.ko 2023-03-27T09:46:13.310 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/async_tx/async_xor.ko 2023-03-27T09:46:13.310 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/async_tx/raid6test.ko 2023-03-27T09:46:13.310 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/authenc.ko 2023-03-27T09:46:13.310 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/authencesn.ko 2023-03-27T09:46:13.310 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/blake2b_generic.ko 2023-03-27T09:46:13.311 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/blowfish_common.ko 2023-03-27T09:46:13.311 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/blowfish_generic.ko 2023-03-27T09:46:13.311 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/camellia_generic.ko 2023-03-27T09:46:13.311 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/cast5_generic.ko 2023-03-27T09:46:13.311 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/cast6_generic.ko 2023-03-27T09:46:13.311 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/cast_common.ko 2023-03-27T09:46:13.312 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/ccm.ko 2023-03-27T09:46:13.312 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/cmac.ko 2023-03-27T09:46:13.312 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/crc32_generic.ko 2023-03-27T09:46:13.312 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/crypto_engine.ko 2023-03-27T09:46:13.312 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/crypto_user.ko 2023-03-27T09:46:13.312 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/cts.ko 2023-03-27T09:46:13.312 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/des_generic.ko 2023-03-27T09:46:13.313 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/dh_generic.ko 2023-03-27T09:46:13.313 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/echainiv.ko 2023-03-27T09:46:13.313 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/essiv.ko 2023-03-27T09:46:13.313 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/fcrypt.ko 2023-03-27T09:46:13.313 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/khazad.ko 2023-03-27T09:46:13.313 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/lz4.ko 2023-03-27T09:46:13.313 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/lz4hc.ko 2023-03-27T09:46:13.314 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/md4.ko 2023-03-27T09:46:13.314 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/michael_mic.ko 2023-03-27T09:46:13.314 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/pcbc.ko 2023-03-27T09:46:13.314 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/pcrypt.ko 2023-03-27T09:46:13.314 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/rmd160.ko 2023-03-27T09:46:13.314 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/seed.ko 2023-03-27T09:46:13.314 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/serpent_generic.ko 2023-03-27T09:46:13.314 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/tcrypt.ko 2023-03-27T09:46:13.315 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/tea.ko 2023-03-27T09:46:13.315 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/twofish_common.ko 2023-03-27T09:46:13.315 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/twofish_generic.ko 2023-03-27T09:46:13.315 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/vmac.ko 2023-03-27T09:46:13.315 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/wp512.ko 2023-03-27T09:46:13.315 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/xcbc.ko 2023-03-27T09:46:13.316 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/xor.ko 2023-03-27T09:46:13.316 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/xxhash_generic.ko 2023-03-27T09:46:13.316 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers 2023-03-27T09:46:13.316 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/acpi 2023-03-27T09:46:13.316 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/acpi/acpi_ipmi.ko 2023-03-27T09:46:13.317 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/acpi/acpi_pad.ko 2023-03-27T09:46:13.317 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/acpi/custom_method.ko 2023-03-27T09:46:13.317 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/acpi/ec_sys.ko 2023-03-27T09:46:13.317 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/acpi/platform_profile.ko 2023-03-27T09:46:13.317 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/acpi/sbs.ko 2023-03-27T09:46:13.318 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/acpi/sbshc.ko 2023-03-27T09:46:13.318 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/acpi/video.ko 2023-03-27T09:46:13.318 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata 2023-03-27T09:46:13.318 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/acard-ahci.ko 2023-03-27T09:46:13.318 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/ahci_platform.ko 2023-03-27T09:46:13.318 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/ata_generic.ko 2023-03-27T09:46:13.318 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/libahci_platform.ko 2023-03-27T09:46:13.319 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_acpi.ko 2023-03-27T09:46:13.319 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_ali.ko 2023-03-27T09:46:13.319 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_amd.ko 2023-03-27T09:46:13.319 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_artop.ko 2023-03-27T09:46:13.319 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_atiixp.ko 2023-03-27T09:46:13.319 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_atp867x.ko 2023-03-27T09:46:13.319 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_cmd640.ko 2023-03-27T09:46:13.319 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_cmd64x.ko 2023-03-27T09:46:13.320 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_cypress.ko 2023-03-27T09:46:13.320 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_efar.ko 2023-03-27T09:46:13.320 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_hpt366.ko 2023-03-27T09:46:13.320 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_hpt37x.ko 2023-03-27T09:46:13.320 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_hpt3x2n.ko 2023-03-27T09:46:13.320 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_hpt3x3.ko 2023-03-27T09:46:13.321 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_it8213.ko 2023-03-27T09:46:13.321 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_it821x.ko 2023-03-27T09:46:13.321 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_jmicron.ko 2023-03-27T09:46:13.321 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_marvell.ko 2023-03-27T09:46:13.321 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_mpiix.ko 2023-03-27T09:46:13.321 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_netcell.ko 2023-03-27T09:46:13.322 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_ninja32.ko 2023-03-27T09:46:13.322 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_ns87410.ko 2023-03-27T09:46:13.322 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_ns87415.ko 2023-03-27T09:46:13.322 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_oldpiix.ko 2023-03-27T09:46:13.322 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_opti.ko 2023-03-27T09:46:13.322 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_optidma.ko 2023-03-27T09:46:13.322 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_pcmcia.ko 2023-03-27T09:46:13.323 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_pdc2027x.ko 2023-03-27T09:46:13.323 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-03-27T09:46:13.323 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_piccolo.ko 2023-03-27T09:46:13.323 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_rdc.ko 2023-03-27T09:46:13.323 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_sch.ko 2023-03-27T09:46:13.323 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_serverworks.ko 2023-03-27T09:46:13.323 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_sil680.ko 2023-03-27T09:46:13.323 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_sis.ko 2023-03-27T09:46:13.324 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_sl82c105.ko 2023-03-27T09:46:13.324 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_triflex.ko 2023-03-27T09:46:13.324 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_via.ko 2023-03-27T09:46:13.324 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pdc_adma.ko 2023-03-27T09:46:13.324 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/sata_inic162x.ko 2023-03-27T09:46:13.325 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/sata_mv.ko 2023-03-27T09:46:13.325 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/sata_nv.ko 2023-03-27T09:46:13.325 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/sata_promise.ko 2023-03-27T09:46:13.325 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/sata_qstor.ko 2023-03-27T09:46:13.325 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/sata_sil.ko 2023-03-27T09:46:13.325 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/sata_sil24.ko 2023-03-27T09:46:13.326 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/sata_sis.ko 2023-03-27T09:46:13.326 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/sata_svw.ko 2023-03-27T09:46:13.326 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/sata_sx4.ko 2023-03-27T09:46:13.326 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/sata_uli.ko 2023-03-27T09:46:13.326 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/sata_via.ko 2023-03-27T09:46:13.326 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/sata_vsc.ko 2023-03-27T09:46:13.326 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/auxdisplay 2023-03-27T09:46:13.326 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/auxdisplay/ks0108.ko 2023-03-27T09:46:13.327 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/base 2023-03-27T09:46:13.327 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/base/regmap 2023-03-27T09:46:13.327 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/base/regmap/regmap-i2c.ko 2023-03-27T09:46:13.327 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/bcma 2023-03-27T09:46:13.327 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/bcma/bcma.ko 2023-03-27T09:46:13.327 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block 2023-03-27T09:46:13.327 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/aoe 2023-03-27T09:46:13.328 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/aoe/aoe.ko 2023-03-27T09:46:13.328 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/brd.ko 2023-03-27T09:46:13.328 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/drbd 2023-03-27T09:46:13.328 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/drbd/drbd.ko 2023-03-27T09:46:13.328 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/floppy.ko 2023-03-27T09:46:13.328 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/loop.ko 2023-03-27T09:46:13.328 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/mtip32xx 2023-03-27T09:46:13.329 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-03-27T09:46:13.329 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/nbd.ko 2023-03-27T09:46:13.329 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/null_blk 2023-03-27T09:46:13.329 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/null_blk/null_blk.ko 2023-03-27T09:46:13.329 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/pktcdvd.ko 2023-03-27T09:46:13.329 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/rbd.ko 2023-03-27T09:46:13.330 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/virtio_blk.ko 2023-03-27T09:46:13.330 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/xen-blkback 2023-03-27T09:46:13.330 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-03-27T09:46:13.330 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/xen-blkfront.ko 2023-03-27T09:46:13.330 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/zram 2023-03-27T09:46:13.330 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/zram/zram.ko 2023-03-27T09:46:13.331 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char 2023-03-27T09:46:13.331 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/hangcheck-timer.ko 2023-03-27T09:46:13.331 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/hw_random 2023-03-27T09:46:13.331 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/hw_random/amd-rng.ko 2023-03-27T09:46:13.331 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/hw_random/intel-rng.ko 2023-03-27T09:46:13.332 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-03-27T09:46:13.332 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/hw_random/via-rng.ko 2023-03-27T09:46:13.332 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/hw_random/virtio-rng.ko 2023-03-27T09:46:13.332 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/ipmi 2023-03-27T09:46:13.332 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-03-27T09:46:13.332 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-03-27T09:46:13.332 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-03-27T09:46:13.333 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/ipmi/ipmi_si.ko 2023-03-27T09:46:13.333 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-03-27T09:46:13.333 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-03-27T09:46:13.333 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/lp.ko 2023-03-27T09:46:13.333 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/mwave 2023-03-27T09:46:13.333 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/mwave/mwave.ko 2023-03-27T09:46:13.334 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/pcmcia 2023-03-27T09:46:13.334 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/pcmcia/cm4000_cs.ko 2023-03-27T09:46:13.334 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/pcmcia/cm4040_cs.ko 2023-03-27T09:46:13.334 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/ppdev.ko 2023-03-27T09:46:13.335 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/tlclk.ko 2023-03-27T09:46:13.335 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/tpm 2023-03-27T09:46:13.335 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/tpm/tpm.ko 2023-03-27T09:46:13.335 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/tpm/tpm_atmel.ko 2023-03-27T09:46:13.336 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/tpm/tpm_infineon.ko 2023-03-27T09:46:13.336 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/tpm/tpm_nsc.ko 2023-03-27T09:46:13.336 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/tpm/tpm_tis.ko 2023-03-27T09:46:13.337 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-03-27T09:46:13.337 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/uv_mmtimer.ko 2023-03-27T09:46:13.337 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/virtio_console.ko 2023-03-27T09:46:13.338 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/cpufreq 2023-03-27T09:46:13.338 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-03-27T09:46:13.338 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-03-27T09:46:13.339 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/cpufreq/p4-clockmod.ko 2023-03-27T09:46:13.339 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-03-27T09:46:13.339 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/cpufreq/powernow-k8.ko 2023-03-27T09:46:13.340 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/cpufreq/speedstep-lib.ko 2023-03-27T09:46:13.340 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/crypto 2023-03-27T09:46:13.340 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/crypto/ccp 2023-03-27T09:46:13.340 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-03-27T09:46:13.341 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/crypto/ccp/ccp.ko 2023-03-27T09:46:13.341 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/crypto/padlock-aes.ko 2023-03-27T09:46:13.341 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/crypto/padlock-sha.ko 2023-03-27T09:46:13.341 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/crypto/qat 2023-03-27T09:46:13.342 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/crypto/qat/qat_common 2023-03-27T09:46:13.342 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/crypto/qat/qat_common/intel_qat.ko 2023-03-27T09:46:13.342 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/crypto/qat/qat_dh895xcc 2023-03-27T09:46:13.342 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/crypto/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-03-27T09:46:13.342 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/crypto/virtio 2023-03-27T09:46:13.342 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-03-27T09:46:13.343 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/dca 2023-03-27T09:46:13.343 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/dca/dca.ko 2023-03-27T09:46:13.343 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/devfreq 2023-03-27T09:46:13.343 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-03-27T09:46:13.343 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/dma 2023-03-27T09:46:13.343 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/dma/dw 2023-03-27T09:46:13.343 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/dma/dw/dw_dmac.ko 2023-03-27T09:46:13.344 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/dma/ioat 2023-03-27T09:46:13.344 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/dma/ioat/ioatdma.ko 2023-03-27T09:46:13.344 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac 2023-03-27T09:46:13.344 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac/amd64_edac.ko 2023-03-27T09:46:13.344 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac/e752x_edac.ko 2023-03-27T09:46:13.344 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac/edac_mce_amd.ko 2023-03-27T09:46:13.344 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac/i3000_edac.ko 2023-03-27T09:46:13.345 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac/i3200_edac.ko 2023-03-27T09:46:13.345 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac/i5100_edac.ko 2023-03-27T09:46:13.345 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac/i5400_edac.ko 2023-03-27T09:46:13.345 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac/i7300_edac.ko 2023-03-27T09:46:13.345 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac/i7core_edac.ko 2023-03-27T09:46:13.345 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac/i82975x_edac.ko 2023-03-27T09:46:13.345 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac/ie31200_edac.ko 2023-03-27T09:46:13.346 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac/sb_edac.ko 2023-03-27T09:46:13.346 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac/x38_edac.ko 2023-03-27T09:46:13.346 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/firmware 2023-03-27T09:46:13.346 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/firmware/edd.ko 2023-03-27T09:46:13.346 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/firmware/iscsi_ibft.ko 2023-03-27T09:46:13.346 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/gpio 2023-03-27T09:46:13.346 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/gpio/gpio-ich.ko 2023-03-27T09:46:13.347 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/gpio/gpio-viperboard.ko 2023-03-27T09:46:13.347 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid 2023-03-27T09:46:13.347 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-appleir.ko 2023-03-27T09:46:13.347 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-aureal.ko 2023-03-27T09:46:13.347 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-axff.ko 2023-03-27T09:46:13.347 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-dr.ko 2023-03-27T09:46:13.347 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-elecom.ko 2023-03-27T09:46:13.347 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-elo.ko 2023-03-27T09:46:13.348 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-emsff.ko 2023-03-27T09:46:13.348 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-gaff.ko 2023-03-27T09:46:13.348 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-gt683r.ko 2023-03-27T09:46:13.348 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-gyration.ko 2023-03-27T09:46:13.348 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-holtek-kbd.ko 2023-03-27T09:46:13.348 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-holtek-mouse.ko 2023-03-27T09:46:13.348 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-holtekff.ko 2023-03-27T09:46:13.348 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-hyperv.ko 2023-03-27T09:46:13.349 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-icade.ko 2023-03-27T09:46:13.349 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-keytouch.ko 2023-03-27T09:46:13.349 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-kye.ko 2023-03-27T09:46:13.349 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-lcpower.ko 2023-03-27T09:46:13.349 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-led.ko 2023-03-27T09:46:13.349 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-lenovo.ko 2023-03-27T09:46:13.350 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-logitech-dj.ko 2023-03-27T09:46:13.350 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-03-27T09:46:13.350 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-multitouch.ko 2023-03-27T09:46:13.350 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-ortek.ko 2023-03-27T09:46:13.350 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-penmount.ko 2023-03-27T09:46:13.350 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-petalynx.ko 2023-03-27T09:46:13.350 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-picolcd.ko 2023-03-27T09:46:13.350 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-pl.ko 2023-03-27T09:46:13.351 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-plantronics.ko 2023-03-27T09:46:13.351 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-primax.ko 2023-03-27T09:46:13.351 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-rmi.ko 2023-03-27T09:46:13.351 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-roccat-arvo.ko 2023-03-27T09:46:13.351 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-roccat-common.ko 2023-03-27T09:46:13.351 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-roccat-isku.ko 2023-03-27T09:46:13.351 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-roccat-kone.ko 2023-03-27T09:46:13.352 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-03-27T09:46:13.352 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-roccat-konepure.ko 2023-03-27T09:46:13.352 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-03-27T09:46:13.352 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-roccat-lua.ko 2023-03-27T09:46:13.352 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-roccat-pyra.ko 2023-03-27T09:46:13.352 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-roccat-ryos.ko 2023-03-27T09:46:13.353 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-roccat-savu.ko 2023-03-27T09:46:13.353 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-roccat.ko 2023-03-27T09:46:13.353 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-saitek.ko 2023-03-27T09:46:13.353 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-samsung.ko 2023-03-27T09:46:13.353 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-sensor-hub.ko 2023-03-27T09:46:13.353 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-sjoy.ko 2023-03-27T09:46:13.353 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-sony.ko 2023-03-27T09:46:13.354 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-speedlink.ko 2023-03-27T09:46:13.354 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-steelseries.ko 2023-03-27T09:46:13.354 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-sunplus.ko 2023-03-27T09:46:13.354 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-thrustmaster.ko 2023-03-27T09:46:13.354 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-tivo.ko 2023-03-27T09:46:13.354 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-tmff.ko 2023-03-27T09:46:13.354 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-topseed.ko 2023-03-27T09:46:13.354 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-twinhan.ko 2023-03-27T09:46:13.355 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-uclogic.ko 2023-03-27T09:46:13.355 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-waltop.ko 2023-03-27T09:46:13.355 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-wiimote.ko 2023-03-27T09:46:13.355 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-xinmo.ko 2023-03-27T09:46:13.355 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-zpff.ko 2023-03-27T09:46:13.355 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-zydacron.ko 2023-03-27T09:46:13.355 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/uhid.ko 2023-03-27T09:46:13.355 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/wacom.ko 2023-03-27T09:46:13.356 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hv 2023-03-27T09:46:13.356 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hv/hv_balloon.ko 2023-03-27T09:46:13.356 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hv/hv_utils.ko 2023-03-27T09:46:13.356 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hv/hv_vmbus.ko 2023-03-27T09:46:13.356 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon 2023-03-27T09:46:13.357 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/abituguru.ko 2023-03-27T09:46:13.357 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/abituguru3.ko 2023-03-27T09:46:13.357 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/acpi_power_meter.ko 2023-03-27T09:46:13.357 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ad7414.ko 2023-03-27T09:46:13.357 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ad7418.ko 2023-03-27T09:46:13.357 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/adc128d818.ko 2023-03-27T09:46:13.357 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/adm1025.ko 2023-03-27T09:46:13.358 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/adm1026.ko 2023-03-27T09:46:13.358 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/adm1029.ko 2023-03-27T09:46:13.358 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/adm1031.ko 2023-03-27T09:46:13.358 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/adm9240.ko 2023-03-27T09:46:13.358 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ads7828.ko 2023-03-27T09:46:13.358 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/adt7410.ko 2023-03-27T09:46:13.358 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/adt7411.ko 2023-03-27T09:46:13.358 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/adt7462.ko 2023-03-27T09:46:13.359 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/adt7470.ko 2023-03-27T09:46:13.359 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/adt7475.ko 2023-03-27T09:46:13.359 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/adt7x10.ko 2023-03-27T09:46:13.359 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/amc6821.ko 2023-03-27T09:46:13.359 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/applesmc.ko 2023-03-27T09:46:13.359 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/asb100.ko 2023-03-27T09:46:13.359 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/asc7621.ko 2023-03-27T09:46:13.360 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/asus_atk0110.ko 2023-03-27T09:46:13.360 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/atxp1.ko 2023-03-27T09:46:13.360 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/coretemp.ko 2023-03-27T09:46:13.360 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-03-27T09:46:13.360 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/dme1737.ko 2023-03-27T09:46:13.360 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ds1621.ko 2023-03-27T09:46:13.360 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ds620.ko 2023-03-27T09:46:13.361 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/emc1403.ko 2023-03-27T09:46:13.361 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/emc6w201.ko 2023-03-27T09:46:13.361 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/f71805f.ko 2023-03-27T09:46:13.361 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/f71882fg.ko 2023-03-27T09:46:13.361 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/f75375s.ko 2023-03-27T09:46:13.361 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/fam15h_power.ko 2023-03-27T09:46:13.361 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/fschmd.ko 2023-03-27T09:46:13.362 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/g760a.ko 2023-03-27T09:46:13.362 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/g762.ko 2023-03-27T09:46:13.362 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/gl518sm.ko 2023-03-27T09:46:13.362 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/gl520sm.ko 2023-03-27T09:46:13.362 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/hwmon-vid.ko 2023-03-27T09:46:13.362 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/i5500_temp.ko 2023-03-27T09:46:13.362 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/i5k_amb.ko 2023-03-27T09:46:13.363 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ibmaem.ko 2023-03-27T09:46:13.363 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ibmpex.ko 2023-03-27T09:46:13.363 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ina209.ko 2023-03-27T09:46:13.363 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ina2xx.ko 2023-03-27T09:46:13.363 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/it87.ko 2023-03-27T09:46:13.364 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/k10temp.ko 2023-03-27T09:46:13.364 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/k8temp.ko 2023-03-27T09:46:13.364 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lineage-pem.ko 2023-03-27T09:46:13.364 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm63.ko 2023-03-27T09:46:13.364 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm73.ko 2023-03-27T09:46:13.364 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm75.ko 2023-03-27T09:46:13.364 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm77.ko 2023-03-27T09:46:13.365 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm78.ko 2023-03-27T09:46:13.365 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm80.ko 2023-03-27T09:46:13.365 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm83.ko 2023-03-27T09:46:13.365 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm85.ko 2023-03-27T09:46:13.365 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm87.ko 2023-03-27T09:46:13.365 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm90.ko 2023-03-27T09:46:13.366 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm92.ko 2023-03-27T09:46:13.366 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm93.ko 2023-03-27T09:46:13.366 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm95234.ko 2023-03-27T09:46:13.366 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm95241.ko 2023-03-27T09:46:13.366 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm95245.ko 2023-03-27T09:46:13.366 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ltc2945.ko 2023-03-27T09:46:13.366 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ltc4151.ko 2023-03-27T09:46:13.367 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ltc4215.ko 2023-03-27T09:46:13.367 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ltc4222.ko 2023-03-27T09:46:13.367 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ltc4245.ko 2023-03-27T09:46:13.367 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ltc4260.ko 2023-03-27T09:46:13.368 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ltc4261.ko 2023-03-27T09:46:13.368 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/max16065.ko 2023-03-27T09:46:13.368 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/max1619.ko 2023-03-27T09:46:13.368 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/max1668.ko 2023-03-27T09:46:13.368 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/max197.ko 2023-03-27T09:46:13.368 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/max6639.ko 2023-03-27T09:46:13.369 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/max6650.ko 2023-03-27T09:46:13.369 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/max6697.ko 2023-03-27T09:46:13.369 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/mcp3021.ko 2023-03-27T09:46:13.369 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/nct6683.ko 2023-03-27T09:46:13.369 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/nct6775-core.ko 2023-03-27T09:46:13.369 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/nct6775.ko 2023-03-27T09:46:13.369 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/nct7802.ko 2023-03-27T09:46:13.370 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pc87360.ko 2023-03-27T09:46:13.370 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pc87427.ko 2023-03-27T09:46:13.370 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pcf8591.ko 2023-03-27T09:46:13.370 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pmbus 2023-03-27T09:46:13.370 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-03-27T09:46:13.370 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-03-27T09:46:13.370 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-03-27T09:46:13.371 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pmbus/max16064.ko 2023-03-27T09:46:13.371 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pmbus/max34440.ko 2023-03-27T09:46:13.371 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pmbus/max8688.ko 2023-03-27T09:46:13.371 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-03-27T09:46:13.371 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-03-27T09:46:13.371 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-03-27T09:46:13.371 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-03-27T09:46:13.371 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-03-27T09:46:13.372 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-03-27T09:46:13.372 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/powr1220.ko 2023-03-27T09:46:13.372 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/sch5627.ko 2023-03-27T09:46:13.372 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/sch5636.ko 2023-03-27T09:46:13.372 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/sch56xx-common.ko 2023-03-27T09:46:13.372 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/sht15.ko 2023-03-27T09:46:13.372 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/sht21.ko 2023-03-27T09:46:13.373 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/shtc1.ko 2023-03-27T09:46:13.373 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/sis5595.ko 2023-03-27T09:46:13.373 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/smsc47b397.ko 2023-03-27T09:46:13.373 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/smsc47m1.ko 2023-03-27T09:46:13.374 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/smsc47m192.ko 2023-03-27T09:46:13.374 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/thmc50.ko 2023-03-27T09:46:13.374 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/tmp102.ko 2023-03-27T09:46:13.374 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/tmp103.ko 2023-03-27T09:46:13.374 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/tmp401.ko 2023-03-27T09:46:13.374 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/tmp421.ko 2023-03-27T09:46:13.375 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/via-cputemp.ko 2023-03-27T09:46:13.375 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/via686a.ko 2023-03-27T09:46:13.375 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/vt1211.ko 2023-03-27T09:46:13.375 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/vt8231.ko 2023-03-27T09:46:13.375 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/w83627ehf.ko 2023-03-27T09:46:13.375 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/w83627hf.ko 2023-03-27T09:46:13.375 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/w83781d.ko 2023-03-27T09:46:13.376 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/w83791d.ko 2023-03-27T09:46:13.376 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/w83792d.ko 2023-03-27T09:46:13.376 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/w83793.ko 2023-03-27T09:46:13.376 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/w83795.ko 2023-03-27T09:46:13.376 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/w83l785ts.ko 2023-03-27T09:46:13.376 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/w83l786ng.ko 2023-03-27T09:46:13.376 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c 2023-03-27T09:46:13.376 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/algos 2023-03-27T09:46:13.377 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-03-27T09:46:13.377 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-03-27T09:46:13.377 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses 2023-03-27T09:46:13.377 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-03-27T09:46:13.378 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-03-27T09:46:13.378 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-03-27T09:46:13.378 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-03-27T09:46:13.379 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-03-27T09:46:13.379 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-03-27T09:46:13.379 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-03-27T09:46:13.379 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-03-27T09:46:13.380 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-i801.ko 2023-03-27T09:46:13.380 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-isch.ko 2023-03-27T09:46:13.380 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-03-27T09:46:13.381 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-03-27T09:46:13.381 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-03-27T09:46:13.381 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-parport.ko 2023-03-27T09:46:13.381 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-03-27T09:46:13.382 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-03-27T09:46:13.382 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-03-27T09:46:13.382 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-03-27T09:46:13.383 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-03-27T09:46:13.383 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-03-27T09:46:13.383 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-via.ko 2023-03-27T09:46:13.383 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-03-27T09:46:13.384 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-03-27T09:46:13.384 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/i2c-dev.ko 2023-03-27T09:46:13.384 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/i2c-mux.ko 2023-03-27T09:46:13.384 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-03-27T09:46:13.384 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/i2c-smbus.ko 2023-03-27T09:46:13.385 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/i2c-stub.ko 2023-03-27T09:46:13.385 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband 2023-03-27T09:46:13.385 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/core 2023-03-27T09:46:13.385 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/core/ib_cm.ko 2023-03-27T09:46:13.385 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/core/ib_core.ko 2023-03-27T09:46:13.385 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/core/ib_umad.ko 2023-03-27T09:46:13.386 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-03-27T09:46:13.386 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/core/iw_cm.ko 2023-03-27T09:46:13.386 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/core/rdma_cm.ko 2023-03-27T09:46:13.386 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-03-27T09:46:13.386 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/hw 2023-03-27T09:46:13.387 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/hw/cxgb4 2023-03-27T09:46:13.387 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-03-27T09:46:13.387 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/hw/mlx4 2023-03-27T09:46:13.387 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-03-27T09:46:13.387 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/hw/mlx5 2023-03-27T09:46:13.387 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-03-27T09:46:13.387 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/hw/mthca 2023-03-27T09:46:13.388 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-03-27T09:46:13.388 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/ulp 2023-03-27T09:46:13.388 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/ulp/ipoib 2023-03-27T09:46:13.388 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-03-27T09:46:13.388 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/ulp/iser 2023-03-27T09:46:13.388 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-03-27T09:46:13.388 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/ulp/isert 2023-03-27T09:46:13.388 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-03-27T09:46:13.389 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/ulp/srp 2023-03-27T09:46:13.389 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-03-27T09:46:13.389 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/ulp/srpt 2023-03-27T09:46:13.389 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-03-27T09:46:13.389 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input 2023-03-27T09:46:13.389 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/gameport 2023-03-27T09:46:13.389 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-03-27T09:46:13.390 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/gameport/fm801-gp.ko 2023-03-27T09:46:13.390 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/gameport/gameport.ko 2023-03-27T09:46:13.390 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/gameport/lightning.ko 2023-03-27T09:46:13.390 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/gameport/ns558.ko 2023-03-27T09:46:13.390 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/joydev.ko 2023-03-27T09:46:13.390 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/keyboard 2023-03-27T09:46:13.390 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/keyboard/gpio_keys.ko 2023-03-27T09:46:13.391 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/mouse 2023-03-27T09:46:13.391 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/mouse/appletouch.ko 2023-03-27T09:46:13.391 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/mouse/bcm5974.ko 2023-03-27T09:46:13.391 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/mouse/cyapatp.ko 2023-03-27T09:46:13.391 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/mouse/elan_i2c.ko 2023-03-27T09:46:13.391 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/mouse/sermouse.ko 2023-03-27T09:46:13.391 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-03-27T09:46:13.392 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/mouse/synaptics_usb.ko 2023-03-27T09:46:13.392 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/mouse/vsxxxaa.ko 2023-03-27T09:46:13.392 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/rmi4 2023-03-27T09:46:13.392 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/rmi4/rmi_core.ko 2023-03-27T09:46:13.392 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/serio 2023-03-27T09:46:13.392 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/serio/altera_ps2.ko 2023-03-27T09:46:13.392 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/serio/arc_ps2.ko 2023-03-27T09:46:13.393 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-03-27T09:46:13.393 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/serio/serio_raw.ko 2023-03-27T09:46:13.393 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/sparse-keymap.ko 2023-03-27T09:46:13.393 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/iommu 2023-03-27T09:46:13.393 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/iommu/amd 2023-03-27T09:46:13.393 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/iommu/amd/iommu_v2.ko 2023-03-27T09:46:13.393 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds 2023-03-27T09:46:13.394 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/leds-blinkm.ko 2023-03-27T09:46:13.394 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/leds-lm3530.ko 2023-03-27T09:46:13.394 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/leds-lp3944.ko 2023-03-27T09:46:13.394 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/leds-ss4200.ko 2023-03-27T09:46:13.394 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/trigger 2023-03-27T09:46:13.394 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-03-27T09:46:13.395 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-03-27T09:46:13.395 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-03-27T09:46:13.395 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-03-27T09:46:13.395 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2023-03-27T09:46:13.395 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-03-27T09:46:13.395 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-03-27T09:46:13.395 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-03-27T09:46:13.396 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-03-27T09:46:13.396 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md 2023-03-27T09:46:13.396 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/bcache 2023-03-27T09:46:13.396 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/bcache/bcache.ko 2023-03-27T09:46:13.396 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-bio-prison.ko 2023-03-27T09:46:13.396 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-cache-smq.ko 2023-03-27T09:46:13.397 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-cache.ko 2023-03-27T09:46:13.397 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-crypt.ko 2023-03-27T09:46:13.397 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-delay.ko 2023-03-27T09:46:13.397 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-flakey.ko 2023-03-27T09:46:13.397 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-log-userspace.ko 2023-03-27T09:46:13.397 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-multipath.ko 2023-03-27T09:46:13.397 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-queue-length.ko 2023-03-27T09:46:13.398 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-raid.ko 2023-03-27T09:46:13.398 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-round-robin.ko 2023-03-27T09:46:13.398 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-service-time.ko 2023-03-27T09:46:13.398 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-switch.ko 2023-03-27T09:46:13.398 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-thin-pool.ko 2023-03-27T09:46:13.398 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-verity.ko 2023-03-27T09:46:13.398 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/faulty.ko 2023-03-27T09:46:13.399 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/linear.ko 2023-03-27T09:46:13.399 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/multipath.ko 2023-03-27T09:46:13.399 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/persistent-data 2023-03-27T09:46:13.399 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-03-27T09:46:13.399 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/raid0.ko 2023-03-27T09:46:13.399 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/raid1.ko 2023-03-27T09:46:13.399 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/raid10.ko 2023-03-27T09:46:13.400 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/raid456.ko 2023-03-27T09:46:13.400 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/message 2023-03-27T09:46:13.400 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/message/fusion 2023-03-27T09:46:13.400 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/message/fusion/mptbase.ko 2023-03-27T09:46:13.400 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/message/fusion/mptctl.ko 2023-03-27T09:46:13.400 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/message/fusion/mptfc.ko 2023-03-27T09:46:13.400 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/message/fusion/mptlan.ko 2023-03-27T09:46:13.400 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/message/fusion/mptsas.ko 2023-03-27T09:46:13.401 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/message/fusion/mptscsih.ko 2023-03-27T09:46:13.401 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/message/fusion/mptspi.ko 2023-03-27T09:46:13.401 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/mfd 2023-03-27T09:46:13.401 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/mfd/lpc_ich.ko 2023-03-27T09:46:13.401 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/mfd/lpc_sch.ko 2023-03-27T09:46:13.401 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/mfd/mfd-core.ko 2023-03-27T09:46:13.401 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/mfd/sm501.ko 2023-03-27T09:46:13.402 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/mfd/viperboard.ko 2023-03-27T09:46:13.402 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/mfd/vx855.ko 2023-03-27T09:46:13.402 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/mfd/wl1273-core.ko 2023-03-27T09:46:13.402 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc 2023-03-27T09:46:13.402 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/altera-stapl 2023-03-27T09:46:13.402 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-03-27T09:46:13.402 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/apds9802als.ko 2023-03-27T09:46:13.402 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/apds990x.ko 2023-03-27T09:46:13.403 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/bh1770glc.ko 2023-03-27T09:46:13.403 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/cb710 2023-03-27T09:46:13.403 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/cb710/cb710.ko 2023-03-27T09:46:13.403 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/eeprom 2023-03-27T09:46:13.403 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/eeprom/at24.ko 2023-03-27T09:46:13.403 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/eeprom/eeprom.ko 2023-03-27T09:46:13.403 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-03-27T09:46:13.404 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/eeprom/max6875.ko 2023-03-27T09:46:13.404 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/enclosure.ko 2023-03-27T09:46:13.404 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/hpilo.ko 2023-03-27T09:46:13.404 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/isl29003.ko 2023-03-27T09:46:13.404 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/isl29020.ko 2023-03-27T09:46:13.404 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/lis3lv02d 2023-03-27T09:46:13.404 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-03-27T09:46:13.405 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-03-27T09:46:13.405 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/mei 2023-03-27T09:46:13.405 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/mei/mei-me.ko 2023-03-27T09:46:13.405 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/mei/mei-txe.ko 2023-03-27T09:46:13.405 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/mei/mei.ko 2023-03-27T09:46:13.406 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/sgi-gru 2023-03-27T09:46:13.406 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/sgi-gru/gru.ko 2023-03-27T09:46:13.406 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/sgi-xp 2023-03-27T09:46:13.406 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/sgi-xp/xp.ko 2023-03-27T09:46:13.406 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/sgi-xp/xpc.ko 2023-03-27T09:46:13.406 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-03-27T09:46:13.406 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/tifm_7xx1.ko 2023-03-27T09:46:13.407 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/tifm_core.ko 2023-03-27T09:46:13.407 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/tsl2550.ko 2023-03-27T09:46:13.407 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/vmw_balloon.ko 2023-03-27T09:46:13.407 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/vmw_vmci 2023-03-27T09:46:13.407 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-03-27T09:46:13.407 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net 2023-03-27T09:46:13.407 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/bonding 2023-03-27T09:46:13.408 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/bonding/bonding.ko 2023-03-27T09:46:13.408 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/dsa 2023-03-27T09:46:13.408 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/dsa/b53 2023-03-27T09:46:13.408 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/dsa/b53/b53_common.ko 2023-03-27T09:46:13.408 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/dsa/bcm-sf2.ko 2023-03-27T09:46:13.408 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/dsa/mv88e6060.ko 2023-03-27T09:46:13.409 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/dsa/mv88e6xxx 2023-03-27T09:46:13.409 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-03-27T09:46:13.409 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/dummy.ko 2023-03-27T09:46:13.409 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/eql.ko 2023-03-27T09:46:13.409 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet 2023-03-27T09:46:13.409 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/3com 2023-03-27T09:46:13.410 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-03-27T09:46:13.410 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-03-27T09:46:13.410 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-03-27T09:46:13.410 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-03-27T09:46:13.410 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/8390 2023-03-27T09:46:13.411 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/8390/8390.ko 2023-03-27T09:46:13.411 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-03-27T09:46:13.411 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-03-27T09:46:13.411 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-03-27T09:46:13.411 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/adaptec 2023-03-27T09:46:13.412 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-03-27T09:46:13.412 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/agere 2023-03-27T09:46:13.412 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/agere/et131x.ko 2023-03-27T09:46:13.412 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/alteon 2023-03-27T09:46:13.413 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-03-27T09:46:13.413 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/altera 2023-03-27T09:46:13.413 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-03-27T09:46:13.413 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/amd 2023-03-27T09:46:13.414 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-03-27T09:46:13.414 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-03-27T09:46:13.414 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-03-27T09:46:13.414 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/atheros 2023-03-27T09:46:13.414 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/atheros/alx 2023-03-27T09:46:13.415 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-03-27T09:46:13.415 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/atheros/atl1c 2023-03-27T09:46:13.415 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-03-27T09:46:13.415 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/atheros/atl1e 2023-03-27T09:46:13.415 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-03-27T09:46:13.416 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/atheros/atlx 2023-03-27T09:46:13.416 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-03-27T09:46:13.416 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-03-27T09:46:13.416 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/broadcom 2023-03-27T09:46:13.417 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-03-27T09:46:13.417 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-03-27T09:46:13.417 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-03-27T09:46:13.418 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-03-27T09:46:13.418 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-03-27T09:46:13.418 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/broadcom/genet 2023-03-27T09:46:13.418 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-03-27T09:46:13.419 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-03-27T09:46:13.419 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/brocade 2023-03-27T09:46:13.419 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/brocade/bna 2023-03-27T09:46:13.419 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-03-27T09:46:13.419 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/chelsio 2023-03-27T09:46:13.420 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/chelsio/cxgb 2023-03-27T09:46:13.420 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-03-27T09:46:13.420 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-03-27T09:46:13.420 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-03-27T09:46:13.420 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-03-27T09:46:13.421 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-03-27T09:46:13.421 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-03-27T09:46:13.421 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-03-27T09:46:13.421 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-03-27T09:46:13.422 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-03-27T09:46:13.422 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/cisco 2023-03-27T09:46:13.422 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/cisco/enic 2023-03-27T09:46:13.422 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-03-27T09:46:13.422 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/dec 2023-03-27T09:46:13.423 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/dec/tulip 2023-03-27T09:46:13.423 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-03-27T09:46:13.423 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-03-27T09:46:13.423 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-03-27T09:46:13.424 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-03-27T09:46:13.424 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-03-27T09:46:13.424 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-03-27T09:46:13.425 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/dlink 2023-03-27T09:46:13.425 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-03-27T09:46:13.425 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-03-27T09:46:13.425 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/dnet.ko 2023-03-27T09:46:13.426 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/emulex 2023-03-27T09:46:13.426 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/emulex/benet 2023-03-27T09:46:13.426 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-03-27T09:46:13.426 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/ethoc.ko 2023-03-27T09:46:13.427 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/fealnx.ko 2023-03-27T09:46:13.427 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel 2023-03-27T09:46:13.427 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/e100.ko 2023-03-27T09:46:13.427 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/e1000 2023-03-27T09:46:13.428 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-03-27T09:46:13.428 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/e1000e 2023-03-27T09:46:13.428 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-03-27T09:46:13.429 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/fm10k 2023-03-27T09:46:13.429 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-03-27T09:46:13.429 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/i40e 2023-03-27T09:46:13.429 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-03-27T09:46:13.430 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/iavf 2023-03-27T09:46:13.430 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-03-27T09:46:13.430 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/igb 2023-03-27T09:46:13.430 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-03-27T09:46:13.431 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/igbvf 2023-03-27T09:46:13.431 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-03-27T09:46:13.431 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/ixgb 2023-03-27T09:46:13.431 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/ixgb/ixgb.ko 2023-03-27T09:46:13.432 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/ixgbe 2023-03-27T09:46:13.432 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-03-27T09:46:13.432 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/ixgbevf 2023-03-27T09:46:13.432 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-03-27T09:46:13.433 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/jme.ko 2023-03-27T09:46:13.433 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/marvell 2023-03-27T09:46:13.433 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-03-27T09:46:13.433 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/marvell/skge.ko 2023-03-27T09:46:13.433 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-03-27T09:46:13.434 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/mellanox 2023-03-27T09:46:13.434 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/mellanox/mlx4 2023-03-27T09:46:13.434 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-03-27T09:46:13.434 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-03-27T09:46:13.434 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/mellanox/mlx5 2023-03-27T09:46:13.435 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-03-27T09:46:13.435 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-03-27T09:46:13.435 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/micrel 2023-03-27T09:46:13.435 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-03-27T09:46:13.436 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/myricom 2023-03-27T09:46:13.436 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/myricom/myri10ge 2023-03-27T09:46:13.436 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-03-27T09:46:13.436 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/natsemi 2023-03-27T09:46:13.437 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-03-27T09:46:13.437 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-03-27T09:46:13.437 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/neterion 2023-03-27T09:46:13.437 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-03-27T09:46:13.438 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/nvidia 2023-03-27T09:46:13.438 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-03-27T09:46:13.438 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/packetengines 2023-03-27T09:46:13.438 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-03-27T09:46:13.439 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-03-27T09:46:13.439 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/qlogic 2023-03-27T09:46:13.439 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/qlogic/netxen 2023-03-27T09:46:13.439 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-03-27T09:46:13.440 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-03-27T09:46:13.440 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-03-27T09:46:13.440 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-03-27T09:46:13.440 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/rdc 2023-03-27T09:46:13.441 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-03-27T09:46:13.441 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/realtek 2023-03-27T09:46:13.441 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-03-27T09:46:13.441 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-03-27T09:46:13.441 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/realtek/atp.ko 2023-03-27T09:46:13.441 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-03-27T09:46:13.442 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/rocker 2023-03-27T09:46:13.442 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-03-27T09:46:13.442 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/sfc 2023-03-27T09:46:13.442 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-03-27T09:46:13.442 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/silan 2023-03-27T09:46:13.442 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-03-27T09:46:13.442 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/sis 2023-03-27T09:46:13.442 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/sis/sis190.ko 2023-03-27T09:46:13.443 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/sis/sis900.ko 2023-03-27T09:46:13.443 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/smsc 2023-03-27T09:46:13.443 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-03-27T09:46:13.443 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-03-27T09:46:13.443 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-03-27T09:46:13.443 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-03-27T09:46:13.443 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/stmicro 2023-03-27T09:46:13.443 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/stmicro/stmmac 2023-03-27T09:46:13.444 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-03-27T09:46:13.444 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-03-27T09:46:13.444 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/sun 2023-03-27T09:46:13.444 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/sun/cassini.ko 2023-03-27T09:46:13.444 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/sun/niu.ko 2023-03-27T09:46:13.444 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/sun/sungem.ko 2023-03-27T09:46:13.444 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-03-27T09:46:13.445 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/tehuti 2023-03-27T09:46:13.445 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-03-27T09:46:13.445 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/ti 2023-03-27T09:46:13.445 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/ti/tlan.ko 2023-03-27T09:46:13.445 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/via 2023-03-27T09:46:13.445 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-03-27T09:46:13.445 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-03-27T09:46:13.445 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/wiznet 2023-03-27T09:46:13.446 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-03-27T09:46:13.446 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-03-27T09:46:13.446 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/xircom 2023-03-27T09:46:13.446 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-03-27T09:46:13.446 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/geneve.ko 2023-03-27T09:46:13.446 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/hyperv 2023-03-27T09:46:13.448 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-03-27T09:46:13.448 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ieee802154 2023-03-27T09:46:13.448 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ieee802154/fakelb.ko 2023-03-27T09:46:13.448 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ifb.ko 2023-03-27T09:46:13.448 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ipvlan 2023-03-27T09:46:13.448 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ipvlan/ipvlan.ko 2023-03-27T09:46:13.449 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/macvlan.ko 2023-03-27T09:46:13.449 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/macvtap.ko 2023-03-27T09:46:13.449 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/mdio 2023-03-27T09:46:13.449 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/mdio.ko 2023-03-27T09:46:13.449 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-03-27T09:46:13.449 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-03-27T09:46:13.449 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/mii.ko 2023-03-27T09:46:13.450 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/net_failover.ko 2023-03-27T09:46:13.450 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/netconsole.ko 2023-03-27T09:46:13.450 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/nlmon.ko 2023-03-27T09:46:13.450 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/pcs 2023-03-27T09:46:13.450 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/pcs/pcs-altera-tse.ko 2023-03-27T09:46:13.450 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-03-27T09:46:13.451 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy 2023-03-27T09:46:13.451 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/amd.ko 2023-03-27T09:46:13.451 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-03-27T09:46:13.451 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-03-27T09:46:13.451 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/bcm7xxx.ko 2023-03-27T09:46:13.451 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/bcm87xx.ko 2023-03-27T09:46:13.451 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/broadcom.ko 2023-03-27T09:46:13.451 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/cicada.ko 2023-03-27T09:46:13.452 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/davicom.ko 2023-03-27T09:46:13.452 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/dp83640.ko 2023-03-27T09:46:13.452 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/et1011c.ko 2023-03-27T09:46:13.452 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/icplus.ko 2023-03-27T09:46:13.452 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/lxt.ko 2023-03-27T09:46:13.452 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/marvell.ko 2023-03-27T09:46:13.452 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/micrel.ko 2023-03-27T09:46:13.452 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/national.ko 2023-03-27T09:46:13.453 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/phylink.ko 2023-03-27T09:46:13.453 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/qsemi.ko 2023-03-27T09:46:13.453 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/realtek.ko 2023-03-27T09:46:13.453 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/smsc.ko 2023-03-27T09:46:13.453 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/ste10Xp.ko 2023-03-27T09:46:13.453 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/vitesse.ko 2023-03-27T09:46:13.453 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ppp 2023-03-27T09:46:13.453 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ppp/bsd_comp.ko 2023-03-27T09:46:13.454 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ppp/ppp_async.ko 2023-03-27T09:46:13.454 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ppp/ppp_deflate.ko 2023-03-27T09:46:13.454 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ppp/ppp_generic.ko 2023-03-27T09:46:13.454 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ppp/ppp_mppe.ko 2023-03-27T09:46:13.454 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ppp/ppp_synctty.ko 2023-03-27T09:46:13.454 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ppp/pppoe.ko 2023-03-27T09:46:13.454 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ppp/pppox.ko 2023-03-27T09:46:13.455 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ppp/pptp.ko 2023-03-27T09:46:13.455 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/slip 2023-03-27T09:46:13.455 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/slip/slhc.ko 2023-03-27T09:46:13.455 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/slip/slip.ko 2023-03-27T09:46:13.455 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/sungem_phy.ko 2023-03-27T09:46:13.455 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/tap.ko 2023-03-27T09:46:13.455 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/team 2023-03-27T09:46:13.456 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/team/team.ko 2023-03-27T09:46:13.456 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/team/team_mode_activebackup.ko 2023-03-27T09:46:13.456 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/team/team_mode_broadcast.ko 2023-03-27T09:46:13.456 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-03-27T09:46:13.456 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/team/team_mode_random.ko 2023-03-27T09:46:13.456 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-03-27T09:46:13.456 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/tun.ko 2023-03-27T09:46:13.456 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/veth.ko 2023-03-27T09:46:13.457 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/virtio_net.ko 2023-03-27T09:46:13.457 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/vmxnet3 2023-03-27T09:46:13.457 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-03-27T09:46:13.457 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/vxlan 2023-03-27T09:46:13.457 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/vxlan/vxlan.ko 2023-03-27T09:46:13.457 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/xen-netback 2023-03-27T09:46:13.457 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/xen-netback/xen-netback.ko 2023-03-27T09:46:13.458 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/xen-netfront.ko 2023-03-27T09:46:13.458 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ntb 2023-03-27T09:46:13.458 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ntb/ntb.ko 2023-03-27T09:46:13.458 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/nvme 2023-03-27T09:46:13.458 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/nvme/host 2023-03-27T09:46:13.458 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/nvme/host/nvme-core.ko 2023-03-27T09:46:13.458 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/nvme/host/nvme.ko 2023-03-27T09:46:13.458 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/parport 2023-03-27T09:46:13.459 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/parport/parport.ko 2023-03-27T09:46:13.459 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/parport/parport_cs.ko 2023-03-27T09:46:13.459 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/parport/parport_pc.ko 2023-03-27T09:46:13.459 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/parport/parport_serial.ko 2023-03-27T09:46:13.459 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pci 2023-03-27T09:46:13.459 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pci/hotplug 2023-03-27T09:46:13.459 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-03-27T09:46:13.460 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pci/pcie 2023-03-27T09:46:13.460 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pci/pcie/aer_inject.ko 2023-03-27T09:46:13.460 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pci/xen-pcifront.ko 2023-03-27T09:46:13.460 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pcmcia 2023-03-27T09:46:13.460 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pcmcia/i82092.ko 2023-03-27T09:46:13.460 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pcmcia/pd6729.ko 2023-03-27T09:46:13.460 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pcmcia/yenta_socket.ko 2023-03-27T09:46:13.460 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pinctrl 2023-03-27T09:46:13.461 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pinctrl/intel 2023-03-27T09:46:13.461 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-03-27T09:46:13.461 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform 2023-03-27T09:46:13.461 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86 2023-03-27T09:46:13.461 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/acer-wmi.ko 2023-03-27T09:46:13.461 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/acerhdf.ko 2023-03-27T09:46:13.461 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/apple-gmux.ko 2023-03-27T09:46:13.461 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/asus-laptop.ko 2023-03-27T09:46:13.462 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-03-27T09:46:13.462 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/asus-wmi.ko 2023-03-27T09:46:13.462 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/classmate-laptop.ko 2023-03-27T09:46:13.462 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-03-27T09:46:13.462 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-03-27T09:46:13.462 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-03-27T09:46:13.462 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-03-27T09:46:13.462 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/hdaps.ko 2023-03-27T09:46:13.463 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/intel 2023-03-27T09:46:13.463 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-03-27T09:46:13.463 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/intel_ips.ko 2023-03-27T09:46:13.463 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/msi-wmi.ko 2023-03-27T09:46:13.463 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/mxm-wmi.ko 2023-03-27T09:46:13.463 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-03-27T09:46:13.463 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/samsung-laptop.ko 2023-03-27T09:46:13.463 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/samsung-q10.ko 2023-03-27T09:46:13.464 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/topstar-laptop.ko 2023-03-27T09:46:13.464 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-03-27T09:46:13.464 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/toshiba_haps.ko 2023-03-27T09:46:13.464 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/wmi-bmof.ko 2023-03-27T09:46:13.464 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/wmi.ko 2023-03-27T09:46:13.464 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/powercap 2023-03-27T09:46:13.464 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/powercap/intel_rapl_common.ko 2023-03-27T09:46:13.465 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/powercap/intel_rapl_msr.ko 2023-03-27T09:46:13.465 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pps 2023-03-27T09:46:13.465 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pps/clients 2023-03-27T09:46:13.465 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pps/clients/pps-gpio.ko 2023-03-27T09:46:13.465 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pps/clients/pps-ldisc.ko 2023-03-27T09:46:13.465 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pps/clients/pps_parport.ko 2023-03-27T09:46:13.465 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pps/pps_core.ko 2023-03-27T09:46:13.465 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ptp 2023-03-27T09:46:13.466 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ptp/ptp.ko 2023-03-27T09:46:13.466 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ptp/ptp_kvm.ko 2023-03-27T09:46:13.466 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc 2023-03-27T09:46:13.466 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-bq32k.ko 2023-03-27T09:46:13.466 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-bq4802.ko 2023-03-27T09:46:13.466 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-ds1286.ko 2023-03-27T09:46:13.466 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-ds1307.ko 2023-03-27T09:46:13.466 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-ds1374.ko 2023-03-27T09:46:13.467 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-ds1511.ko 2023-03-27T09:46:13.467 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-ds1553.ko 2023-03-27T09:46:13.467 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-ds1672.ko 2023-03-27T09:46:13.467 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-ds1742.ko 2023-03-27T09:46:13.467 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-ds2404.ko 2023-03-27T09:46:13.467 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-ds3232.ko 2023-03-27T09:46:13.467 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-em3027.ko 2023-03-27T09:46:13.467 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-fm3130.ko 2023-03-27T09:46:13.468 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-isl12022.ko 2023-03-27T09:46:13.468 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-isl1208.ko 2023-03-27T09:46:13.468 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-m41t80.ko 2023-03-27T09:46:13.468 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-m48t35.ko 2023-03-27T09:46:13.468 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-m48t59.ko 2023-03-27T09:46:13.468 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-max6900.ko 2023-03-27T09:46:13.468 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-msm6242.ko 2023-03-27T09:46:13.468 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-pcf2127.ko 2023-03-27T09:46:13.469 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-pcf85063.ko 2023-03-27T09:46:13.469 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-pcf8523.ko 2023-03-27T09:46:13.469 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-pcf8563.ko 2023-03-27T09:46:13.469 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-pcf8583.ko 2023-03-27T09:46:13.469 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-rp5c01.ko 2023-03-27T09:46:13.469 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-rs5c372.ko 2023-03-27T09:46:13.469 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-03-27T09:46:13.470 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-rx8025.ko 2023-03-27T09:46:13.470 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-rx8581.ko 2023-03-27T09:46:13.470 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-03-27T09:46:13.470 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-x1205.ko 2023-03-27T09:46:13.470 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi 2023-03-27T09:46:13.470 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/3w-9xxx.ko 2023-03-27T09:46:13.470 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/3w-sas.ko 2023-03-27T09:46:13.470 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/3w-xxxx.ko 2023-03-27T09:46:13.471 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/BusLogic.ko 2023-03-27T09:46:13.471 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/a100u2w.ko 2023-03-27T09:46:13.471 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/aacraid 2023-03-27T09:46:13.471 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/aacraid/aacraid.ko 2023-03-27T09:46:13.471 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/advansys.ko 2023-03-27T09:46:13.471 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/aic7xxx 2023-03-27T09:46:13.471 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-03-27T09:46:13.472 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-03-27T09:46:13.472 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/am53c974.ko 2023-03-27T09:46:13.472 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/arcmsr 2023-03-27T09:46:13.472 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-03-27T09:46:13.472 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/atp870u.ko 2023-03-27T09:46:13.472 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/be2iscsi 2023-03-27T09:46:13.472 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-03-27T09:46:13.472 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/bfa 2023-03-27T09:46:13.473 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/bfa/bfa.ko 2023-03-27T09:46:13.473 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/bnx2fc 2023-03-27T09:46:13.473 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-03-27T09:46:13.473 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/bnx2i 2023-03-27T09:46:13.473 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-03-27T09:46:13.473 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/ch.ko 2023-03-27T09:46:13.473 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/csiostor 2023-03-27T09:46:13.474 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/csiostor/csiostor.ko 2023-03-27T09:46:13.474 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/cxgbi 2023-03-27T09:46:13.474 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/cxgbi/cxgb3i 2023-03-27T09:46:13.474 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-03-27T09:46:13.474 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/cxgbi/cxgb4i 2023-03-27T09:46:13.474 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-03-27T09:46:13.474 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-03-27T09:46:13.474 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/dc395x.ko 2023-03-27T09:46:13.475 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/device_handler 2023-03-27T09:46:13.475 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-03-27T09:46:13.475 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-03-27T09:46:13.475 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-03-27T09:46:13.475 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-03-27T09:46:13.475 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/dmx3191d.ko 2023-03-27T09:46:13.475 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/esas2r 2023-03-27T09:46:13.475 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/esas2r/esas2r.ko 2023-03-27T09:46:13.476 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/esp_scsi.ko 2023-03-27T09:46:13.476 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/fcoe 2023-03-27T09:46:13.476 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/fcoe/fcoe.ko 2023-03-27T09:46:13.476 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-03-27T09:46:13.476 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/fnic 2023-03-27T09:46:13.476 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/fnic/fnic.ko 2023-03-27T09:46:13.476 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/hpsa.ko 2023-03-27T09:46:13.476 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/hptiop.ko 2023-03-27T09:46:13.477 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/hv_storvsc.ko 2023-03-27T09:46:13.477 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/initio.ko 2023-03-27T09:46:13.477 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/ipr.ko 2023-03-27T09:46:13.477 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/ips.ko 2023-03-27T09:46:13.477 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/isci 2023-03-27T09:46:13.477 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/isci/isci.ko 2023-03-27T09:46:13.477 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-03-27T09:46:13.478 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/iscsi_tcp.ko 2023-03-27T09:46:13.478 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/libfc 2023-03-27T09:46:13.478 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/libfc/libfc.ko 2023-03-27T09:46:13.478 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/libiscsi.ko 2023-03-27T09:46:13.478 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/libiscsi_tcp.ko 2023-03-27T09:46:13.478 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/libsas 2023-03-27T09:46:13.478 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/libsas/libsas.ko 2023-03-27T09:46:13.478 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/lpfc 2023-03-27T09:46:13.479 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/lpfc/lpfc.ko 2023-03-27T09:46:13.479 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/megaraid 2023-03-27T09:46:13.479 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/megaraid.ko 2023-03-27T09:46:13.479 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-03-27T09:46:13.479 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-03-27T09:46:13.479 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-03-27T09:46:13.479 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/mpt3sas 2023-03-27T09:46:13.479 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-03-27T09:46:13.480 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/mvsas 2023-03-27T09:46:13.480 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/mvsas/mvsas.ko 2023-03-27T09:46:13.480 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/mvumi.ko 2023-03-27T09:46:13.480 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/pm8001 2023-03-27T09:46:13.480 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-03-27T09:46:13.480 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/pmcraid.ko 2023-03-27T09:46:13.480 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/qla1280.ko 2023-03-27T09:46:13.480 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/qla2xxx 2023-03-27T09:46:13.481 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-03-27T09:46:13.481 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-03-27T09:46:13.481 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/qla4xxx 2023-03-27T09:46:13.481 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-03-27T09:46:13.481 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/raid_class.ko 2023-03-27T09:46:13.481 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/scsi_debug.ko 2023-03-27T09:46:13.481 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/scsi_transport_fc.ko 2023-03-27T09:46:13.481 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-03-27T09:46:13.482 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/scsi_transport_sas.ko 2023-03-27T09:46:13.482 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/scsi_transport_spi.ko 2023-03-27T09:46:13.482 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/scsi_transport_srp.ko 2023-03-27T09:46:13.482 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/ses.ko 2023-03-27T09:46:13.482 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/st.ko 2023-03-27T09:46:13.482 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/stex.ko 2023-03-27T09:46:13.482 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/sym53c8xx_2 2023-03-27T09:46:13.482 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-03-27T09:46:13.483 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/virtio_scsi.ko 2023-03-27T09:46:13.483 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/vmw_pvscsi.ko 2023-03-27T09:46:13.483 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/wd719x.ko 2023-03-27T09:46:13.483 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ssb 2023-03-27T09:46:13.483 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ssb/ssb.ko 2023-03-27T09:46:13.483 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/target 2023-03-27T09:46:13.483 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/target/iscsi 2023-03-27T09:46:13.483 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-03-27T09:46:13.484 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/target/loopback 2023-03-27T09:46:13.484 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/target/loopback/tcm_loop.ko 2023-03-27T09:46:13.484 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/target/target_core_file.ko 2023-03-27T09:46:13.484 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/target/target_core_iblock.ko 2023-03-27T09:46:13.484 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/target/target_core_mod.ko 2023-03-27T09:46:13.484 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/target/target_core_pscsi.ko 2023-03-27T09:46:13.484 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/target/target_core_user.ko 2023-03-27T09:46:13.484 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/target/tcm_fc 2023-03-27T09:46:13.485 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-03-27T09:46:13.485 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal 2023-03-27T09:46:13.485 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel 2023-03-27T09:46:13.485 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/int340x_thermal 2023-03-27T09:46:13.485 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-03-27T09:46:13.485 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-03-27T09:46:13.485 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-03-27T09:46:13.485 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-03-27T09:46:13.486 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-03-27T09:46:13.486 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-03-27T09:46:13.486 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-03-27T09:46:13.486 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-03-27T09:46:13.486 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-03-27T09:46:13.486 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-03-27T09:46:13.486 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-03-27T09:46:13.487 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-03-27T09:46:13.487 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-03-27T09:46:13.487 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-03-27T09:46:13.487 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-03-27T09:46:13.487 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/tty 2023-03-27T09:46:13.487 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/tty/ipwireless 2023-03-27T09:46:13.487 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-03-27T09:46:13.487 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/tty/n_gsm.ko 2023-03-27T09:46:13.488 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/tty/n_hdlc.ko 2023-03-27T09:46:13.488 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/tty/nozomi.ko 2023-03-27T09:46:13.488 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/tty/serial 2023-03-27T09:46:13.488 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/tty/serial/8250 2023-03-27T09:46:13.488 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-03-27T09:46:13.488 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/tty/serial/arc_uart.ko 2023-03-27T09:46:13.488 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/tty/serial/jsm 2023-03-27T09:46:13.488 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/tty/serial/jsm/jsm.ko 2023-03-27T09:46:13.489 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/tty/synclink_gt.ko 2023-03-27T09:46:13.489 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ufs 2023-03-27T09:46:13.489 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ufs/core 2023-03-27T09:46:13.489 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ufs/core/ufshcd-core.ko 2023-03-27T09:46:13.489 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ufs/host 2023-03-27T09:46:13.489 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-03-27T09:46:13.489 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/uio 2023-03-27T09:46:13.490 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/uio/uio.ko 2023-03-27T09:46:13.490 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/uio/uio_aec.ko 2023-03-27T09:46:13.490 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/uio/uio_cif.ko 2023-03-27T09:46:13.490 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/uio/uio_pci_generic.ko 2023-03-27T09:46:13.490 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/uio/uio_sercos3.ko 2023-03-27T09:46:13.490 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb 2023-03-27T09:46:13.490 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/class 2023-03-27T09:46:13.490 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/class/cdc-acm.ko 2023-03-27T09:46:13.491 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/class/cdc-wdm.ko 2023-03-27T09:46:13.491 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/class/usblp.ko 2023-03-27T09:46:13.491 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/class/usbtmc.ko 2023-03-27T09:46:13.491 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/host 2023-03-27T09:46:13.491 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/host/sl811-hcd.ko 2023-03-27T09:46:13.491 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/host/u132-hcd.ko 2023-03-27T09:46:13.491 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/image 2023-03-27T09:46:13.491 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/image/mdc800.ko 2023-03-27T09:46:13.492 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/image/microtek.ko 2023-03-27T09:46:13.492 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc 2023-03-27T09:46:13.492 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/adutux.ko 2023-03-27T09:46:13.492 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/appledisplay.ko 2023-03-27T09:46:13.492 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/emi26.ko 2023-03-27T09:46:13.492 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/emi62.ko 2023-03-27T09:46:13.492 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/ezusb.ko 2023-03-27T09:46:13.492 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/ftdi-elan.ko 2023-03-27T09:46:13.493 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/idmouse.ko 2023-03-27T09:46:13.493 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/iowarrior.ko 2023-03-27T09:46:13.493 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/isight_firmware.ko 2023-03-27T09:46:13.493 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/ldusb.ko 2023-03-27T09:46:13.493 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/legousbtower.ko 2023-03-27T09:46:13.493 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/sisusbvga 2023-03-27T09:46:13.493 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-03-27T09:46:13.493 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/trancevibrator.ko 2023-03-27T09:46:13.494 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/usb3503.ko 2023-03-27T09:46:13.494 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/usblcd.ko 2023-03-27T09:46:13.494 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/usbsevseg.ko 2023-03-27T09:46:13.494 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/uss720.ko 2023-03-27T09:46:13.494 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/yurex.ko 2023-03-27T09:46:13.494 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/phy 2023-03-27T09:46:13.494 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/phy/phy-generic.ko 2023-03-27T09:46:13.494 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial 2023-03-27T09:46:13.495 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/aircable.ko 2023-03-27T09:46:13.495 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/ark3116.ko 2023-03-27T09:46:13.495 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/belkin_sa.ko 2023-03-27T09:46:13.495 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/ch341.ko 2023-03-27T09:46:13.495 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/cp210x.ko 2023-03-27T09:46:13.495 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/cyberjack.ko 2023-03-27T09:46:13.495 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/cypress_m8.ko 2023-03-27T09:46:13.495 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/digi_acceleport.ko 2023-03-27T09:46:13.496 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/empeg.ko 2023-03-27T09:46:13.496 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/ftdi_sio.ko 2023-03-27T09:46:13.496 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/garmin_gps.ko 2023-03-27T09:46:13.496 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/io_edgeport.ko 2023-03-27T09:46:13.496 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/io_ti.ko 2023-03-27T09:46:13.496 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/ipaq.ko 2023-03-27T09:46:13.496 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/ipw.ko 2023-03-27T09:46:13.496 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/ir-usb.ko 2023-03-27T09:46:13.497 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-03-27T09:46:13.497 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/keyspan.ko 2023-03-27T09:46:13.497 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/keyspan_pda.ko 2023-03-27T09:46:13.497 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/kl5kusb105.ko 2023-03-27T09:46:13.497 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/kobil_sct.ko 2023-03-27T09:46:13.497 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/mct_u232.ko 2023-03-27T09:46:13.497 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/mos7720.ko 2023-03-27T09:46:13.498 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/mos7840.ko 2023-03-27T09:46:13.498 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/navman.ko 2023-03-27T09:46:13.498 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/omninet.ko 2023-03-27T09:46:13.498 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/opticon.ko 2023-03-27T09:46:13.498 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/option.ko 2023-03-27T09:46:13.498 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/oti6858.ko 2023-03-27T09:46:13.498 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/pl2303.ko 2023-03-27T09:46:13.498 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/qcaux.ko 2023-03-27T09:46:13.499 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/qcserial.ko 2023-03-27T09:46:13.499 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/quatech2.ko 2023-03-27T09:46:13.499 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/safe_serial.ko 2023-03-27T09:46:13.499 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/sierra.ko 2023-03-27T09:46:13.499 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/spcp8x5.ko 2023-03-27T09:46:13.499 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/ssu100.ko 2023-03-27T09:46:13.499 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/symbolserial.ko 2023-03-27T09:46:13.499 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-03-27T09:46:13.500 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-03-27T09:46:13.500 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/usb_debug.ko 2023-03-27T09:46:13.500 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/usb_wwan.ko 2023-03-27T09:46:13.500 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/visor.ko 2023-03-27T09:46:13.500 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/whiteheat.ko 2023-03-27T09:46:13.500 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/xsens_mt.ko 2023-03-27T09:46:13.500 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage 2023-03-27T09:46:13.500 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/uas.ko 2023-03-27T09:46:13.501 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/ums-alauda.ko 2023-03-27T09:46:13.501 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/ums-cypress.ko 2023-03-27T09:46:13.501 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/ums-datafab.ko 2023-03-27T09:46:13.501 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-03-27T09:46:13.501 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/ums-freecom.ko 2023-03-27T09:46:13.501 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/ums-isd200.ko 2023-03-27T09:46:13.501 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-03-27T09:46:13.502 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/ums-karma.ko 2023-03-27T09:46:13.502 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/ums-onetouch.ko 2023-03-27T09:46:13.502 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/ums-realtek.ko 2023-03-27T09:46:13.502 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/ums-sddr09.ko 2023-03-27T09:46:13.502 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/ums-sddr55.ko 2023-03-27T09:46:13.502 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/ums-usbat.ko 2023-03-27T09:46:13.502 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/usb-storage.ko 2023-03-27T09:46:13.502 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/usbip 2023-03-27T09:46:13.503 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/usbip/usbip-core.ko 2023-03-27T09:46:13.503 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/usbip/usbip-host.ko 2023-03-27T09:46:13.503 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-03-27T09:46:13.503 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/vfio 2023-03-27T09:46:13.503 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/vfio/pci 2023-03-27T09:46:13.503 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-03-27T09:46:13.503 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/vfio/pci/vfio-pci.ko 2023-03-27T09:46:13.503 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/vfio/vfio.ko 2023-03-27T09:46:13.504 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-03-27T09:46:13.504 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/vhost 2023-03-27T09:46:13.504 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/vhost/vhost.ko 2023-03-27T09:46:13.504 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/vhost/vhost_iotlb.ko 2023-03-27T09:46:13.504 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/vhost/vhost_net.ko 2023-03-27T09:46:13.504 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/vhost/vhost_scsi.ko 2023-03-27T09:46:13.504 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/video 2023-03-27T09:46:13.504 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/video/backlight 2023-03-27T09:46:13.505 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/video/backlight/apple_bl.ko 2023-03-27T09:46:13.505 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/video/backlight/lcd.ko 2023-03-27T09:46:13.505 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/video/backlight/platform_lcd.ko 2023-03-27T09:46:13.505 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/virtio 2023-03-27T09:46:13.505 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/virtio/virtio.ko 2023-03-27T09:46:13.505 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/virtio/virtio_balloon.ko 2023-03-27T09:46:13.505 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/virtio/virtio_mmio.ko 2023-03-27T09:46:13.505 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/virtio/virtio_pci.ko 2023-03-27T09:46:13.506 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-03-27T09:46:13.506 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-03-27T09:46:13.506 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/virtio/virtio_ring.ko 2023-03-27T09:46:13.506 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1 2023-03-27T09:46:13.506 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/masters 2023-03-27T09:46:13.506 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/masters/ds2482.ko 2023-03-27T09:46:13.506 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/masters/ds2490.ko 2023-03-27T09:46:13.506 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/slaves 2023-03-27T09:46:13.507 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-03-27T09:46:13.507 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-03-27T09:46:13.507 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-03-27T09:46:13.507 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-03-27T09:46:13.507 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-03-27T09:46:13.507 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-03-27T09:46:13.507 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-03-27T09:46:13.507 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-03-27T09:46:13.508 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-03-27T09:46:13.508 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/slaves/w1_smem.ko 2023-03-27T09:46:13.508 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/slaves/w1_therm.ko 2023-03-27T09:46:13.508 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/wire.ko 2023-03-27T09:46:13.508 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog 2023-03-27T09:46:13.508 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/alim1535_wdt.ko 2023-03-27T09:46:13.508 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/alim7101_wdt.ko 2023-03-27T09:46:13.508 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/f71808e_wdt.ko 2023-03-27T09:46:13.509 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/hpwdt.ko 2023-03-27T09:46:13.509 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/i6300esb.ko 2023-03-27T09:46:13.509 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-03-27T09:46:13.509 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/iTCO_wdt.ko 2023-03-27T09:46:13.509 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/ib700wdt.ko 2023-03-27T09:46:13.509 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/ibmasr.ko 2023-03-27T09:46:13.509 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-03-27T09:46:13.510 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/it8712f_wdt.ko 2023-03-27T09:46:13.510 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/it87_wdt.ko 2023-03-27T09:46:13.510 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/machzwd.ko 2023-03-27T09:46:13.510 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/nv_tco.ko 2023-03-27T09:46:13.510 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/pcwd_pci.ko 2023-03-27T09:46:13.510 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/pcwd_usb.ko 2023-03-27T09:46:13.510 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-03-27T09:46:13.510 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/sch311x_wdt.ko 2023-03-27T09:46:13.511 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/softdog.ko 2023-03-27T09:46:13.511 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/sp5100_tco.ko 2023-03-27T09:46:13.511 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/via_wdt.ko 2023-03-27T09:46:13.511 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-03-27T09:46:13.511 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/w83877f_wdt.ko 2023-03-27T09:46:13.511 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/w83977f_wdt.ko 2023-03-27T09:46:13.511 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/wdt_pci.ko 2023-03-27T09:46:13.511 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/xen_wdt.ko 2023-03-27T09:46:13.512 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/xen 2023-03-27T09:46:13.512 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/xen/xen-acpi-processor.ko 2023-03-27T09:46:13.512 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/xen/xen-evtchn.ko 2023-03-27T09:46:13.512 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/xen/xen-gntalloc.ko 2023-03-27T09:46:13.512 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/xen/xen-gntdev.ko 2023-03-27T09:46:13.512 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/xen/xen-pciback 2023-03-27T09:46:13.512 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-03-27T09:46:13.513 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/xen/xen-privcmd.ko 2023-03-27T09:46:13.513 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/xen/xenfs 2023-03-27T09:46:13.513 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/xen/xenfs/xenfs.ko 2023-03-27T09:46:13.513 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs 2023-03-27T09:46:13.513 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/9p 2023-03-27T09:46:13.513 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/9p/9p.ko 2023-03-27T09:46:13.513 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/affs 2023-03-27T09:46:13.514 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/affs/affs.ko 2023-03-27T09:46:13.514 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/befs 2023-03-27T09:46:13.514 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/befs/befs.ko 2023-03-27T09:46:13.514 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/binfmt_misc.ko 2023-03-27T09:46:13.514 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/btrfs 2023-03-27T09:46:13.514 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/btrfs/btrfs.ko 2023-03-27T09:46:13.514 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/cachefiles 2023-03-27T09:46:13.514 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/cachefiles/cachefiles.ko 2023-03-27T09:46:13.515 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ceph 2023-03-27T09:46:13.515 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ceph/ceph.ko 2023-03-27T09:46:13.515 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/cifs 2023-03-27T09:46:13.515 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/cifs/cifs.ko 2023-03-27T09:46:13.515 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/coda 2023-03-27T09:46:13.515 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/coda/coda.ko 2023-03-27T09:46:13.515 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/cramfs 2023-03-27T09:46:13.515 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/cramfs/cramfs.ko 2023-03-27T09:46:13.516 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/dlm 2023-03-27T09:46:13.516 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/dlm/dlm.ko 2023-03-27T09:46:13.516 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ecryptfs 2023-03-27T09:46:13.516 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ecryptfs/ecryptfs.ko 2023-03-27T09:46:13.516 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/f2fs 2023-03-27T09:46:13.516 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/f2fs/f2fs.ko 2023-03-27T09:46:13.516 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/fat 2023-03-27T09:46:13.517 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/fat/fat.ko 2023-03-27T09:46:13.517 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/fat/msdos.ko 2023-03-27T09:46:13.517 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/fat/vfat.ko 2023-03-27T09:46:13.517 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/fscache 2023-03-27T09:46:13.517 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/fscache/fscache.ko 2023-03-27T09:46:13.517 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/fuse 2023-03-27T09:46:13.517 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/fuse/cuse.ko 2023-03-27T09:46:13.517 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/fuse/fuse.ko 2023-03-27T09:46:13.518 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/gfs2 2023-03-27T09:46:13.518 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/gfs2/gfs2.ko 2023-03-27T09:46:13.518 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/hfs 2023-03-27T09:46:13.518 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/hfs/hfs.ko 2023-03-27T09:46:13.518 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/hfsplus 2023-03-27T09:46:13.518 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/hfsplus/hfsplus.ko 2023-03-27T09:46:13.518 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/isofs 2023-03-27T09:46:13.518 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/isofs/isofs.ko 2023-03-27T09:46:13.519 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/jfs 2023-03-27T09:46:13.519 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/jfs/jfs.ko 2023-03-27T09:46:13.519 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/lockd 2023-03-27T09:46:13.519 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/lockd/lockd.ko 2023-03-27T09:46:13.519 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/minix 2023-03-27T09:46:13.519 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/minix/minix.ko 2023-03-27T09:46:13.519 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/netfs 2023-03-27T09:46:13.519 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/netfs/netfs.ko 2023-03-27T09:46:13.520 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfs 2023-03-27T09:46:13.520 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfs/blocklayout 2023-03-27T09:46:13.520 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-03-27T09:46:13.520 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfs/filelayout 2023-03-27T09:46:13.520 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-03-27T09:46:13.520 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfs/flexfilelayout 2023-03-27T09:46:13.520 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-03-27T09:46:13.521 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfs/nfs.ko 2023-03-27T09:46:13.521 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfs/nfsv3.ko 2023-03-27T09:46:13.521 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfs/nfsv4.ko 2023-03-27T09:46:13.521 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfs_common 2023-03-27T09:46:13.521 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfs_common/grace.ko 2023-03-27T09:46:13.521 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfs_common/nfs_acl.ko 2023-03-27T09:46:13.521 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfsd 2023-03-27T09:46:13.521 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfsd/nfsd.ko 2023-03-27T09:46:13.522 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nilfs2 2023-03-27T09:46:13.522 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nilfs2/nilfs2.ko 2023-03-27T09:46:13.522 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls 2023-03-27T09:46:13.522 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/mac-celtic.ko 2023-03-27T09:46:13.522 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/mac-centeuro.ko 2023-03-27T09:46:13.522 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/mac-croatian.ko 2023-03-27T09:46:13.522 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/mac-cyrillic.ko 2023-03-27T09:46:13.523 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/mac-gaelic.ko 2023-03-27T09:46:13.523 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/mac-greek.ko 2023-03-27T09:46:13.523 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/mac-iceland.ko 2023-03-27T09:46:13.523 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/mac-inuit.ko 2023-03-27T09:46:13.523 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/mac-roman.ko 2023-03-27T09:46:13.523 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/mac-romanian.ko 2023-03-27T09:46:13.523 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/mac-turkish.ko 2023-03-27T09:46:13.523 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp1250.ko 2023-03-27T09:46:13.524 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp1251.ko 2023-03-27T09:46:13.524 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp1255.ko 2023-03-27T09:46:13.524 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp737.ko 2023-03-27T09:46:13.524 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp775.ko 2023-03-27T09:46:13.524 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp850.ko 2023-03-27T09:46:13.524 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp852.ko 2023-03-27T09:46:13.524 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp855.ko 2023-03-27T09:46:13.524 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp857.ko 2023-03-27T09:46:13.525 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp860.ko 2023-03-27T09:46:13.525 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp861.ko 2023-03-27T09:46:13.525 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp862.ko 2023-03-27T09:46:13.525 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp863.ko 2023-03-27T09:46:13.525 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp864.ko 2023-03-27T09:46:13.525 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp865.ko 2023-03-27T09:46:13.525 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp866.ko 2023-03-27T09:46:13.525 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp869.ko 2023-03-27T09:46:13.526 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp874.ko 2023-03-27T09:46:13.526 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp932.ko 2023-03-27T09:46:13.526 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp936.ko 2023-03-27T09:46:13.526 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp949.ko 2023-03-27T09:46:13.526 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp950.ko 2023-03-27T09:46:13.526 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_euc-jp.ko 2023-03-27T09:46:13.526 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_iso8859-1.ko 2023-03-27T09:46:13.526 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_iso8859-13.ko 2023-03-27T09:46:13.527 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_iso8859-14.ko 2023-03-27T09:46:13.527 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_iso8859-15.ko 2023-03-27T09:46:13.527 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_iso8859-2.ko 2023-03-27T09:46:13.527 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_iso8859-3.ko 2023-03-27T09:46:13.527 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_iso8859-4.ko 2023-03-27T09:46:13.527 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_iso8859-5.ko 2023-03-27T09:46:13.527 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_iso8859-6.ko 2023-03-27T09:46:13.527 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_iso8859-7.ko 2023-03-27T09:46:13.528 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_iso8859-9.ko 2023-03-27T09:46:13.528 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_koi8-r.ko 2023-03-27T09:46:13.528 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_koi8-ru.ko 2023-03-27T09:46:13.528 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_koi8-u.ko 2023-03-27T09:46:13.528 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_utf8.ko 2023-03-27T09:46:13.528 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ocfs2 2023-03-27T09:46:13.528 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ocfs2/cluster 2023-03-27T09:46:13.528 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-03-27T09:46:13.529 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ocfs2/dlm 2023-03-27T09:46:13.529 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-03-27T09:46:13.529 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ocfs2/dlmfs 2023-03-27T09:46:13.529 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-03-27T09:46:13.529 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ocfs2/ocfs2.ko 2023-03-27T09:46:13.529 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-03-27T09:46:13.530 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-03-27T09:46:13.530 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-03-27T09:46:13.530 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/overlayfs 2023-03-27T09:46:13.530 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/overlayfs/overlay.ko 2023-03-27T09:46:13.530 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/pstore 2023-03-27T09:46:13.530 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/pstore/ramoops.ko 2023-03-27T09:46:13.530 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/reiserfs 2023-03-27T09:46:13.531 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/reiserfs/reiserfs.ko 2023-03-27T09:46:13.531 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/romfs 2023-03-27T09:46:13.531 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/romfs/romfs.ko 2023-03-27T09:46:13.531 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/smbfs_common 2023-03-27T09:46:13.531 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/smbfs_common/cifs_arc4.ko 2023-03-27T09:46:13.531 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/smbfs_common/cifs_md4.ko 2023-03-27T09:46:13.531 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/squashfs 2023-03-27T09:46:13.531 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/squashfs/squashfs.ko 2023-03-27T09:46:13.532 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/sysv 2023-03-27T09:46:13.532 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/sysv/sysv.ko 2023-03-27T09:46:13.532 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/udf 2023-03-27T09:46:13.532 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/udf/udf.ko 2023-03-27T09:46:13.532 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ufs 2023-03-27T09:46:13.532 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ufs/ufs.ko 2023-03-27T09:46:13.532 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/xfs 2023-03-27T09:46:13.532 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/xfs/xfs.ko 2023-03-27T09:46:13.533 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/kernel 2023-03-27T09:46:13.533 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/kernel/rcu 2023-03-27T09:46:13.533 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/kernel/rcu/rcutorture.ko 2023-03-27T09:46:13.533 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/kernel/torture.ko 2023-03-27T09:46:13.533 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/kernel/trace 2023-03-27T09:46:13.533 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-03-27T09:46:13.533 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib 2023-03-27T09:46:13.534 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/asn1_encoder.ko 2023-03-27T09:46:13.534 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/crc-itu-t.ko 2023-03-27T09:46:13.534 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/crc8.ko 2023-03-27T09:46:13.534 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/crypto 2023-03-27T09:46:13.534 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/crypto/libarc4.ko 2023-03-27T09:46:13.534 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/crypto/libdes.ko 2023-03-27T09:46:13.534 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/libcrc32c.ko 2023-03-27T09:46:13.534 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/lru_cache.ko 2023-03-27T09:46:13.535 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/lz4 2023-03-27T09:46:13.535 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/lz4/lz4_compress.ko 2023-03-27T09:46:13.535 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/lz4/lz4hc_compress.ko 2023-03-27T09:46:13.535 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/math 2023-03-27T09:46:13.535 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/math/cordic.ko 2023-03-27T09:46:13.535 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/raid6 2023-03-27T09:46:13.535 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/raid6/raid6_pq.ko 2023-03-27T09:46:13.535 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/reed_solomon 2023-03-27T09:46:13.536 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/reed_solomon/reed_solomon.ko 2023-03-27T09:46:13.536 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/ts_bm.ko 2023-03-27T09:46:13.536 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/ts_fsm.ko 2023-03-27T09:46:13.536 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/ts_kmp.ko 2023-03-27T09:46:13.536 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/zstd 2023-03-27T09:46:13.536 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/zstd/zstd_compress.ko 2023-03-27T09:46:13.536 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/mm 2023-03-27T09:46:13.536 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/mm/hwpoison-inject.ko 2023-03-27T09:46:13.537 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net 2023-03-27T09:46:13.537 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/6lowpan 2023-03-27T09:46:13.537 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/6lowpan/6lowpan.ko 2023-03-27T09:46:13.537 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/6lowpan/nhc_dest.ko 2023-03-27T09:46:13.537 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/6lowpan/nhc_fragment.ko 2023-03-27T09:46:13.537 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/6lowpan/nhc_hop.ko 2023-03-27T09:46:13.537 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/6lowpan/nhc_ipv6.ko 2023-03-27T09:46:13.538 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/6lowpan/nhc_mobility.ko 2023-03-27T09:46:13.538 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/6lowpan/nhc_routing.ko 2023-03-27T09:46:13.538 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/6lowpan/nhc_udp.ko 2023-03-27T09:46:13.538 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/802 2023-03-27T09:46:13.538 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/802/garp.ko 2023-03-27T09:46:13.538 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/802/mrp.ko 2023-03-27T09:46:13.538 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/802/p8022.ko 2023-03-27T09:46:13.538 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/802/psnap.ko 2023-03-27T09:46:13.539 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/802/stp.ko 2023-03-27T09:46:13.539 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/8021q 2023-03-27T09:46:13.539 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/8021q/8021q.ko 2023-03-27T09:46:13.539 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/9p 2023-03-27T09:46:13.539 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/9p/9pnet.ko 2023-03-27T09:46:13.539 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/9p/9pnet_fd.ko 2023-03-27T09:46:13.539 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/9p/9pnet_rdma.ko 2023-03-27T09:46:13.539 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/9p/9pnet_virtio.ko 2023-03-27T09:46:13.540 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/batman-adv 2023-03-27T09:46:13.540 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/batman-adv/batman-adv.ko 2023-03-27T09:46:13.540 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge 2023-03-27T09:46:13.540 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/br_netfilter.ko 2023-03-27T09:46:13.540 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/bridge.ko 2023-03-27T09:46:13.540 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter 2023-03-27T09:46:13.540 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-03-27T09:46:13.540 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_among.ko 2023-03-27T09:46:13.541 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_arp.ko 2023-03-27T09:46:13.541 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-03-27T09:46:13.541 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-03-27T09:46:13.541 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_ip.ko 2023-03-27T09:46:13.541 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-03-27T09:46:13.542 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_limit.ko 2023-03-27T09:46:13.542 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_log.ko 2023-03-27T09:46:13.542 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_mark.ko 2023-03-27T09:46:13.542 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-03-27T09:46:13.542 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-03-27T09:46:13.542 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-03-27T09:46:13.542 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-03-27T09:46:13.542 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_snat.ko 2023-03-27T09:46:13.543 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_stp.ko 2023-03-27T09:46:13.543 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-03-27T09:46:13.543 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-03-27T09:46:13.543 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-03-27T09:46:13.543 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-03-27T09:46:13.543 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebtables.ko 2023-03-27T09:46:13.543 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-03-27T09:46:13.544 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-03-27T09:46:13.544 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-03-27T09:46:13.544 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ceph 2023-03-27T09:46:13.544 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ceph/libceph.ko 2023-03-27T09:46:13.544 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/core 2023-03-27T09:46:13.544 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/core/failover.ko 2023-03-27T09:46:13.545 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/core/pktgen.ko 2023-03-27T09:46:13.545 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/dccp 2023-03-27T09:46:13.545 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/dccp/dccp.ko 2023-03-27T09:46:13.545 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/dccp/dccp_diag.ko 2023-03-27T09:46:13.545 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/dccp/dccp_ipv4.ko 2023-03-27T09:46:13.545 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/dccp/dccp_ipv6.ko 2023-03-27T09:46:13.545 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/dns_resolver 2023-03-27T09:46:13.546 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/dns_resolver/dns_resolver.ko 2023-03-27T09:46:13.546 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/dsa 2023-03-27T09:46:13.546 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/dsa/dsa_core.ko 2023-03-27T09:46:13.546 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/dsa/tag_brcm.ko 2023-03-27T09:46:13.546 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/dsa/tag_dsa.ko 2023-03-27T09:46:13.546 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/dsa/tag_none.ko 2023-03-27T09:46:13.546 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/dsa/tag_trailer.ko 2023-03-27T09:46:13.546 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ieee802154 2023-03-27T09:46:13.547 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ieee802154/6lowpan 2023-03-27T09:46:13.547 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-03-27T09:46:13.547 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ieee802154/ieee802154.ko 2023-03-27T09:46:13.547 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ieee802154/ieee802154_socket.ko 2023-03-27T09:46:13.547 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4 2023-03-27T09:46:13.547 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/ah4.ko 2023-03-27T09:46:13.547 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/esp4.ko 2023-03-27T09:46:13.548 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/fou.ko 2023-03-27T09:46:13.548 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/gre.ko 2023-03-27T09:46:13.548 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/inet_diag.ko 2023-03-27T09:46:13.548 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/ip_gre.ko 2023-03-27T09:46:13.548 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/ip_tunnel.ko 2023-03-27T09:46:13.548 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/ip_vti.ko 2023-03-27T09:46:13.549 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/ipcomp.ko 2023-03-27T09:46:13.549 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/ipip.ko 2023-03-27T09:46:13.549 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter 2023-03-27T09:46:13.549 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/arp_tables.ko 2023-03-27T09:46:13.549 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-03-27T09:46:13.549 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-03-27T09:46:13.549 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-03-27T09:46:13.549 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-03-27T09:46:13.550 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-03-27T09:46:13.550 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-03-27T09:46:13.550 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-03-27T09:46:13.550 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-03-27T09:46:13.550 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-03-27T09:46:13.550 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/iptable_security.ko 2023-03-27T09:46:13.550 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-03-27T09:46:13.551 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-03-27T09:46:13.551 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-03-27T09:46:13.551 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-03-27T09:46:13.551 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-03-27T09:46:13.551 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-03-27T09:46:13.551 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-03-27T09:46:13.551 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-03-27T09:46:13.552 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-03-27T09:46:13.552 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-03-27T09:46:13.552 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tcp_bic.ko 2023-03-27T09:46:13.552 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tcp_dctcp.ko 2023-03-27T09:46:13.552 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tcp_diag.ko 2023-03-27T09:46:13.552 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tcp_highspeed.ko 2023-03-27T09:46:13.552 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tcp_htcp.ko 2023-03-27T09:46:13.552 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tcp_hybla.ko 2023-03-27T09:46:13.553 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tcp_illinois.ko 2023-03-27T09:46:13.553 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tcp_lp.ko 2023-03-27T09:46:13.553 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tcp_scalable.ko 2023-03-27T09:46:13.553 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tcp_vegas.ko 2023-03-27T09:46:13.553 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tcp_veno.ko 2023-03-27T09:46:13.553 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tcp_westwood.ko 2023-03-27T09:46:13.553 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tcp_yeah.ko 2023-03-27T09:46:13.554 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tunnel4.ko 2023-03-27T09:46:13.554 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/udp_diag.ko 2023-03-27T09:46:13.554 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/udp_tunnel.ko 2023-03-27T09:46:13.554 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/xfrm4_tunnel.ko 2023-03-27T09:46:13.554 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6 2023-03-27T09:46:13.554 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/ah6.ko 2023-03-27T09:46:13.554 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/esp6.ko 2023-03-27T09:46:13.554 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/fou6.ko 2023-03-27T09:46:13.555 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/ip6_tunnel.ko 2023-03-27T09:46:13.555 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-03-27T09:46:13.555 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/ip6_vti.ko 2023-03-27T09:46:13.555 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/ipcomp6.ko 2023-03-27T09:46:13.555 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter 2023-03-27T09:46:13.555 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-03-27T09:46:13.555 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-03-27T09:46:13.555 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-03-27T09:46:13.556 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-03-27T09:46:13.556 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-03-27T09:46:13.556 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-03-27T09:46:13.556 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-03-27T09:46:13.556 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-03-27T09:46:13.556 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-03-27T09:46:13.557 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-03-27T09:46:13.557 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-03-27T09:46:13.557 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-03-27T09:46:13.557 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-03-27T09:46:13.557 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-03-27T09:46:13.557 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-03-27T09:46:13.557 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-03-27T09:46:13.558 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-03-27T09:46:13.558 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-03-27T09:46:13.558 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-03-27T09:46:13.558 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-03-27T09:46:13.558 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-03-27T09:46:13.558 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-03-27T09:46:13.558 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-03-27T09:46:13.558 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-03-27T09:46:13.559 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/sit.ko 2023-03-27T09:46:13.559 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/tunnel6.ko 2023-03-27T09:46:13.559 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/xfrm6_tunnel.ko 2023-03-27T09:46:13.559 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/key 2023-03-27T09:46:13.559 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/key/af_key.ko 2023-03-27T09:46:13.559 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/l2tp 2023-03-27T09:46:13.559 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/l2tp/l2tp_core.ko 2023-03-27T09:46:13.559 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/l2tp/l2tp_debugfs.ko 2023-03-27T09:46:13.560 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/l2tp/l2tp_eth.ko 2023-03-27T09:46:13.560 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/l2tp/l2tp_ip.ko 2023-03-27T09:46:13.560 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/l2tp/l2tp_ip6.ko 2023-03-27T09:46:13.560 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/l2tp/l2tp_netlink.ko 2023-03-27T09:46:13.560 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/l2tp/l2tp_ppp.ko 2023-03-27T09:46:13.560 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/llc 2023-03-27T09:46:13.560 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/llc/llc.ko 2023-03-27T09:46:13.560 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/mac802154 2023-03-27T09:46:13.561 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/mac802154/mac802154.ko 2023-03-27T09:46:13.561 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/mpls 2023-03-27T09:46:13.561 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/mpls/mpls_gso.ko 2023-03-27T09:46:13.561 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter 2023-03-27T09:46:13.561 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset 2023-03-27T09:46:13.561 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set.ko 2023-03-27T09:46:13.561 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-03-27T09:46:13.561 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-03-27T09:46:13.562 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-03-27T09:46:13.562 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-03-27T09:46:13.562 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-03-27T09:46:13.562 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-03-27T09:46:13.562 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-03-27T09:46:13.562 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-03-27T09:46:13.562 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-03-27T09:46:13.562 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-03-27T09:46:13.563 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-03-27T09:46:13.563 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-03-27T09:46:13.563 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-03-27T09:46:13.563 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-03-27T09:46:13.563 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-03-27T09:46:13.563 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs 2023-03-27T09:46:13.563 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs.ko 2023-03-27T09:46:13.564 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-03-27T09:46:13.564 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-03-27T09:46:13.564 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-03-27T09:46:13.564 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-03-27T09:46:13.564 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-03-27T09:46:13.564 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-03-27T09:46:13.564 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-03-27T09:46:13.564 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-03-27T09:46:13.565 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-03-27T09:46:13.565 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-03-27T09:46:13.565 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-03-27T09:46:13.565 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-03-27T09:46:13.565 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-03-27T09:46:13.565 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conncount.ko 2023-03-27T09:46:13.565 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conntrack.ko 2023-03-27T09:46:13.566 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-03-27T09:46:13.566 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-03-27T09:46:13.566 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-03-27T09:46:13.566 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conntrack_h323.ko 2023-03-27T09:46:13.566 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conntrack_irc.ko 2023-03-27T09:46:13.566 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-03-27T09:46:13.566 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-03-27T09:46:13.566 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-03-27T09:46:13.567 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conntrack_sane.ko 2023-03-27T09:46:13.567 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conntrack_sip.ko 2023-03-27T09:46:13.567 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-03-27T09:46:13.567 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-03-27T09:46:13.567 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_dup_netdev.ko 2023-03-27T09:46:13.567 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_flow_table.ko 2023-03-27T09:46:13.567 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_flow_table_inet.ko 2023-03-27T09:46:13.567 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_log_syslog.ko 2023-03-27T09:46:13.568 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_nat.ko 2023-03-27T09:46:13.568 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_nat_amanda.ko 2023-03-27T09:46:13.568 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_nat_ftp.ko 2023-03-27T09:46:13.568 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_nat_irc.ko 2023-03-27T09:46:13.568 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_nat_sip.ko 2023-03-27T09:46:13.568 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_nat_tftp.ko 2023-03-27T09:46:13.568 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_synproxy_core.ko 2023-03-27T09:46:13.568 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_tables.ko 2023-03-27T09:46:13.569 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nfnetlink.ko 2023-03-27T09:46:13.569 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nfnetlink_acct.ko 2023-03-27T09:46:13.569 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nfnetlink_log.ko 2023-03-27T09:46:13.569 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nfnetlink_osf.ko 2023-03-27T09:46:13.569 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nfnetlink_queue.ko 2023-03-27T09:46:13.569 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_chain_nat.ko 2023-03-27T09:46:13.569 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_compat.ko 2023-03-27T09:46:13.570 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_ct.ko 2023-03-27T09:46:13.570 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_dup_netdev.ko 2023-03-27T09:46:13.570 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_fib.ko 2023-03-27T09:46:13.570 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_fwd_netdev.ko 2023-03-27T09:46:13.570 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_hash.ko 2023-03-27T09:46:13.570 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_limit.ko 2023-03-27T09:46:13.570 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_log.ko 2023-03-27T09:46:13.570 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_masq.ko 2023-03-27T09:46:13.571 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_nat.ko 2023-03-27T09:46:13.571 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_numgen.ko 2023-03-27T09:46:13.571 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_queue.ko 2023-03-27T09:46:13.571 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_quota.ko 2023-03-27T09:46:13.571 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_redir.ko 2023-03-27T09:46:13.571 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_reject.ko 2023-03-27T09:46:13.571 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_reject_inet.ko 2023-03-27T09:46:13.571 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_socket.ko 2023-03-27T09:46:13.572 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_tunnel.ko 2023-03-27T09:46:13.572 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_AUDIT.ko 2023-03-27T09:46:13.572 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_CHECKSUM.ko 2023-03-27T09:46:13.572 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_CLASSIFY.ko 2023-03-27T09:46:13.572 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-03-27T09:46:13.572 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_CT.ko 2023-03-27T09:46:13.572 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_DSCP.ko 2023-03-27T09:46:13.573 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_HL.ko 2023-03-27T09:46:13.573 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_HMARK.ko 2023-03-27T09:46:13.573 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_IDLETIMER.ko 2023-03-27T09:46:13.573 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_LED.ko 2023-03-27T09:46:13.573 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_LOG.ko 2023-03-27T09:46:13.574 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_MASQUERADE.ko 2023-03-27T09:46:13.574 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_NETMAP.ko 2023-03-27T09:46:13.574 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_NFLOG.ko 2023-03-27T09:46:13.574 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_NFQUEUE.ko 2023-03-27T09:46:13.574 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_RATEEST.ko 2023-03-27T09:46:13.575 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_REDIRECT.ko 2023-03-27T09:46:13.575 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_SECMARK.ko 2023-03-27T09:46:13.575 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_TCPMSS.ko 2023-03-27T09:46:13.575 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-03-27T09:46:13.575 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_TEE.ko 2023-03-27T09:46:13.575 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_TPROXY.ko 2023-03-27T09:46:13.575 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_TRACE.ko 2023-03-27T09:46:13.576 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_addrtype.ko 2023-03-27T09:46:13.576 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_bpf.ko 2023-03-27T09:46:13.576 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_cgroup.ko 2023-03-27T09:46:13.576 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_cluster.ko 2023-03-27T09:46:13.576 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_comment.ko 2023-03-27T09:46:13.576 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_connbytes.ko 2023-03-27T09:46:13.576 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_connlabel.ko 2023-03-27T09:46:13.576 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_connlimit.ko 2023-03-27T09:46:13.577 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_connmark.ko 2023-03-27T09:46:13.577 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_conntrack.ko 2023-03-27T09:46:13.577 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_cpu.ko 2023-03-27T09:46:13.577 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_dccp.ko 2023-03-27T09:46:13.577 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_devgroup.ko 2023-03-27T09:46:13.577 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_dscp.ko 2023-03-27T09:46:13.577 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_ecn.ko 2023-03-27T09:46:13.577 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_esp.ko 2023-03-27T09:46:13.578 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_hashlimit.ko 2023-03-27T09:46:13.578 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_helper.ko 2023-03-27T09:46:13.578 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_hl.ko 2023-03-27T09:46:13.578 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_ipcomp.ko 2023-03-27T09:46:13.578 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_iprange.ko 2023-03-27T09:46:13.578 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_ipvs.ko 2023-03-27T09:46:13.578 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_l2tp.ko 2023-03-27T09:46:13.578 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_length.ko 2023-03-27T09:46:13.579 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_limit.ko 2023-03-27T09:46:13.579 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_mac.ko 2023-03-27T09:46:13.579 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_mark.ko 2023-03-27T09:46:13.579 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_multiport.ko 2023-03-27T09:46:13.579 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_nat.ko 2023-03-27T09:46:13.579 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_nfacct.ko 2023-03-27T09:46:13.579 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_osf.ko 2023-03-27T09:46:13.580 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_owner.ko 2023-03-27T09:46:13.580 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_physdev.ko 2023-03-27T09:46:13.580 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_pkttype.ko 2023-03-27T09:46:13.580 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_policy.ko 2023-03-27T09:46:13.580 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_quota.ko 2023-03-27T09:46:13.580 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_rateest.ko 2023-03-27T09:46:13.580 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_realm.ko 2023-03-27T09:46:13.580 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_recent.ko 2023-03-27T09:46:13.581 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_sctp.ko 2023-03-27T09:46:13.581 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_set.ko 2023-03-27T09:46:13.581 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_socket.ko 2023-03-27T09:46:13.581 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_state.ko 2023-03-27T09:46:13.581 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_statistic.ko 2023-03-27T09:46:13.581 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_string.ko 2023-03-27T09:46:13.581 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_tcpmss.ko 2023-03-27T09:46:13.582 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_time.ko 2023-03-27T09:46:13.582 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_u32.ko 2023-03-27T09:46:13.582 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netlink 2023-03-27T09:46:13.582 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netlink/netlink_diag.ko 2023-03-27T09:46:13.582 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/nsh 2023-03-27T09:46:13.582 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/nsh/nsh.ko 2023-03-27T09:46:13.582 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/openvswitch 2023-03-27T09:46:13.582 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/openvswitch/openvswitch.ko 2023-03-27T09:46:13.583 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/openvswitch/vport-geneve.ko 2023-03-27T09:46:13.583 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/openvswitch/vport-gre.ko 2023-03-27T09:46:13.583 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/openvswitch/vport-vxlan.ko 2023-03-27T09:46:13.583 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/packet 2023-03-27T09:46:13.583 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/packet/af_packet_diag.ko 2023-03-27T09:46:13.583 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/rds 2023-03-27T09:46:13.583 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/rds/rds.ko 2023-03-27T09:46:13.583 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/rds/rds_rdma.ko 2023-03-27T09:46:13.584 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/rds/rds_tcp.ko 2023-03-27T09:46:13.584 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched 2023-03-27T09:46:13.584 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/act_csum.ko 2023-03-27T09:46:13.584 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/act_gact.ko 2023-03-27T09:46:13.584 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/act_ipt.ko 2023-03-27T09:46:13.584 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/act_mirred.ko 2023-03-27T09:46:13.584 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/act_nat.ko 2023-03-27T09:46:13.585 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/act_pedit.ko 2023-03-27T09:46:13.585 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/act_police.ko 2023-03-27T09:46:13.585 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/act_simple.ko 2023-03-27T09:46:13.585 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/act_skbedit.ko 2023-03-27T09:46:13.585 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/act_vlan.ko 2023-03-27T09:46:13.585 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/cls_basic.ko 2023-03-27T09:46:13.585 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/cls_bpf.ko 2023-03-27T09:46:13.585 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/cls_flow.ko 2023-03-27T09:46:13.586 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/cls_fw.ko 2023-03-27T09:46:13.586 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/cls_route.ko 2023-03-27T09:46:13.586 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/cls_u32.ko 2023-03-27T09:46:13.586 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/em_cmp.ko 2023-03-27T09:46:13.586 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/em_ipset.ko 2023-03-27T09:46:13.586 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/em_meta.ko 2023-03-27T09:46:13.586 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/em_nbyte.ko 2023-03-27T09:46:13.586 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/em_text.ko 2023-03-27T09:46:13.587 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/em_u32.ko 2023-03-27T09:46:13.587 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_choke.ko 2023-03-27T09:46:13.587 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_codel.ko 2023-03-27T09:46:13.587 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_drr.ko 2023-03-27T09:46:13.587 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_fq.ko 2023-03-27T09:46:13.587 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_gred.ko 2023-03-27T09:46:13.587 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_hfsc.ko 2023-03-27T09:46:13.588 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_hhf.ko 2023-03-27T09:46:13.588 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_htb.ko 2023-03-27T09:46:13.588 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_ingress.ko 2023-03-27T09:46:13.588 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_mqprio.ko 2023-03-27T09:46:13.588 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_mqprio_lib.ko 2023-03-27T09:46:13.588 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_multiq.ko 2023-03-27T09:46:13.588 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_netem.ko 2023-03-27T09:46:13.588 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_pie.ko 2023-03-27T09:46:13.589 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_plug.ko 2023-03-27T09:46:13.589 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_prio.ko 2023-03-27T09:46:13.589 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_qfq.ko 2023-03-27T09:46:13.589 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_red.ko 2023-03-27T09:46:13.589 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_sfb.ko 2023-03-27T09:46:13.589 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_sfq.ko 2023-03-27T09:46:13.589 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_tbf.ko 2023-03-27T09:46:13.589 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_teql.ko 2023-03-27T09:46:13.590 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sctp 2023-03-27T09:46:13.590 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sctp/sctp.ko 2023-03-27T09:46:13.590 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sctp/sctp_diag.ko 2023-03-27T09:46:13.590 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sunrpc 2023-03-27T09:46:13.590 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sunrpc/auth_gss 2023-03-27T09:46:13.590 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-03-27T09:46:13.590 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-03-27T09:46:13.590 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sunrpc/sunrpc.ko 2023-03-27T09:46:13.591 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sunrpc/xprtrdma 2023-03-27T09:46:13.591 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-03-27T09:46:13.591 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/tipc 2023-03-27T09:46:13.591 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/tipc/diag.ko 2023-03-27T09:46:13.591 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/tipc/tipc.ko 2023-03-27T09:46:13.592 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/unix 2023-03-27T09:46:13.592 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/unix/unix_diag.ko 2023-03-27T09:46:13.592 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/vmw_vsock 2023-03-27T09:46:13.592 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-03-27T09:46:13.592 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-03-27T09:46:13.592 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/vmw_vsock/vsock.ko 2023-03-27T09:46:13.592 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/vmw_vsock/vsock_diag.ko 2023-03-27T09:46:13.592 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/vmw_vsock/vsock_loopback.ko 2023-03-27T09:46:13.593 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/xfrm 2023-03-27T09:46:13.593 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/xfrm/xfrm_ipcomp.ko 2023-03-27T09:46:13.593 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/security 2023-03-27T09:46:13.593 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/security/keys 2023-03-27T09:46:13.593 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/security/keys/encrypted-keys 2023-03-27T09:46:13.593 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-03-27T09:46:13.593 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/security/keys/trusted-keys 2023-03-27T09:46:13.594 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/security/keys/trusted-keys/trusted.ko 2023-03-27T09:46:13.594 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/virt 2023-03-27T09:46:13.594 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/virt/lib 2023-03-27T09:46:13.594 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/virt/lib/irqbypass.ko 2023-03-27T09:46:13.594 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/modules.alias 2023-03-27T09:46:13.594 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/modules.alias.bin 2023-03-27T09:46:13.594 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/modules.builtin 2023-03-27T09:46:13.594 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/modules.builtin.bin 2023-03-27T09:46:13.595 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/modules.builtin.modinfo 2023-03-27T09:46:13.595 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/modules.dep 2023-03-27T09:46:13.595 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/modules.dep.bin 2023-03-27T09:46:13.595 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/modules.devname 2023-03-27T09:46:13.595 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/modules.order 2023-03-27T09:46:13.595 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/modules.softdep 2023-03-27T09:46:13.595 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/modules.symbols 2023-03-27T09:46:13.595 INFO:teuthology.orchestra.run.smithi023.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/modules.symbols.bin 2023-03-27T09:46:13.596 DEBUG:teuthology.task.kernel:get_image_version: 6.3.0-rc3-g1776e8f494e9 2023-03-27T09:46:13.597 DEBUG:teuthology.orchestra.run.smithi023:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-03-27T09:46:13.625 INFO:teuthology.orchestra.run.smithi023.stdout:postinstall scriptlet (using /bin/sh): 2023-03-27T09:46:13.625 INFO:teuthology.orchestra.run.smithi023.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.3.0-rc3-g1776e8f494e9 -a -r /boot/System.map-6.3.0-rc3-g1776e8f494e9 ]; then 2023-03-27T09:46:13.626 INFO:teuthology.orchestra.run.smithi023.stdout:cp /boot/vmlinuz-6.3.0-rc3-g1776e8f494e9 /boot/.vmlinuz-6.3.0-rc3-g1776e8f494e9-rpm 2023-03-27T09:46:13.626 INFO:teuthology.orchestra.run.smithi023.stdout:cp /boot/System.map-6.3.0-rc3-g1776e8f494e9 /boot/.System.map-6.3.0-rc3-g1776e8f494e9-rpm 2023-03-27T09:46:13.626 INFO:teuthology.orchestra.run.smithi023.stdout:rm -f /boot/vmlinuz-6.3.0-rc3-g1776e8f494e9 /boot/System.map-6.3.0-rc3-g1776e8f494e9 2023-03-27T09:46:13.626 INFO:teuthology.orchestra.run.smithi023.stdout:/sbin/installkernel 6.3.0-rc3-g1776e8f494e9 /boot/.vmlinuz-6.3.0-rc3-g1776e8f494e9-rpm /boot/.System.map-6.3.0-rc3-g1776e8f494e9-rpm 2023-03-27T09:46:13.626 INFO:teuthology.orchestra.run.smithi023.stdout:rm -f /boot/.vmlinuz-6.3.0-rc3-g1776e8f494e9-rpm /boot/.System.map-6.3.0-rc3-g1776e8f494e9-rpm 2023-03-27T09:46:13.626 INFO:teuthology.orchestra.run.smithi023.stdout:fi 2023-03-27T09:46:13.626 INFO:teuthology.orchestra.run.smithi023.stdout:preuninstall scriptlet (using /bin/sh): 2023-03-27T09:46:13.627 INFO:teuthology.orchestra.run.smithi023.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-03-27T09:46:13.627 INFO:teuthology.orchestra.run.smithi023.stdout:new-kernel-pkg --remove 6.3.0-rc3-g1776e8f494e9 --rminitrd --initrdfile=/boot/initramfs-6.3.0-rc3-g1776e8f494e9.img 2023-03-27T09:46:13.627 INFO:teuthology.orchestra.run.smithi023.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-03-27T09:46:13.627 INFO:teuthology.orchestra.run.smithi023.stdout:kernel-install remove 6.3.0-rc3-g1776e8f494e9 2023-03-27T09:46:13.627 INFO:teuthology.orchestra.run.smithi023.stdout:fi 2023-03-27T09:46:13.627 INFO:teuthology.orchestra.run.smithi023.stdout:postuninstall scriptlet (using /bin/sh): 2023-03-27T09:46:13.627 INFO:teuthology.orchestra.run.smithi023.stdout:if [ -x /sbin/update-bootloader ]; then 2023-03-27T09:46:13.627 INFO:teuthology.orchestra.run.smithi023.stdout:/sbin/update-bootloader --remove 6.3.0-rc3-g1776e8f494e9 2023-03-27T09:46:13.628 INFO:teuthology.orchestra.run.smithi023.stdout:fi 2023-03-27T09:46:13.629 DEBUG:teuthology.orchestra.run.smithi023:> sudo rpm -qi grub2-tools 2023-03-27T09:46:13.708 INFO:teuthology.orchestra.run.smithi023.stdout:Name : grub2-tools 2023-03-27T09:46:13.708 INFO:teuthology.orchestra.run.smithi023.stdout:Epoch : 1 2023-03-27T09:46:13.709 INFO:teuthology.orchestra.run.smithi023.stdout:Version : 2.02 2023-03-27T09:46:13.709 INFO:teuthology.orchestra.run.smithi023.stdout:Release : 123.el8 2023-03-27T09:46:13.709 INFO:teuthology.orchestra.run.smithi023.stdout:Architecture: x86_64 2023-03-27T09:46:13.709 INFO:teuthology.orchestra.run.smithi023.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-03-27T09:46:13.709 INFO:teuthology.orchestra.run.smithi023.stdout:Group : System Environment/Base 2023-03-27T09:46:13.709 INFO:teuthology.orchestra.run.smithi023.stdout:Size : 9265450 2023-03-27T09:46:13.709 INFO:teuthology.orchestra.run.smithi023.stdout:License : GPLv3+ 2023-03-27T09:46:13.710 INFO:teuthology.orchestra.run.smithi023.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-03-27T09:46:13.710 INFO:teuthology.orchestra.run.smithi023.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-03-27T09:46:13.710 INFO:teuthology.orchestra.run.smithi023.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-03-27T09:46:13.710 INFO:teuthology.orchestra.run.smithi023.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-03-27T09:46:13.710 INFO:teuthology.orchestra.run.smithi023.stdout:Relocations : (not relocatable) 2023-03-27T09:46:13.710 INFO:teuthology.orchestra.run.smithi023.stdout:Packager : Red Hat, Inc. 2023-03-27T09:46:13.710 INFO:teuthology.orchestra.run.smithi023.stdout:Vendor : Red Hat, Inc. 2023-03-27T09:46:13.710 INFO:teuthology.orchestra.run.smithi023.stdout:URL : http://www.gnu.org/software/grub/ 2023-03-27T09:46:13.711 INFO:teuthology.orchestra.run.smithi023.stdout:Summary : Support tools for GRUB. 2023-03-27T09:46:13.711 INFO:teuthology.orchestra.run.smithi023.stdout:Description : 2023-03-27T09:46:13.711 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T09:46:13.711 INFO:teuthology.orchestra.run.smithi023.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-03-27T09:46:13.711 INFO:teuthology.orchestra.run.smithi023.stdout:customizable bootloader with modular architecture. It supports a rich 2023-03-27T09:46:13.711 INFO:teuthology.orchestra.run.smithi023.stdout:variety of kernel formats, file systems, computer architectures and 2023-03-27T09:46:13.711 INFO:teuthology.orchestra.run.smithi023.stdout:hardware devices. 2023-03-27T09:46:13.712 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T09:46:13.712 INFO:teuthology.orchestra.run.smithi023.stdout:This subpackage provides tools for support of all platforms. 2023-03-27T09:46:13.713 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-03-27T09:46:13.713 INFO:teuthology.task.kernel:Updating grub on smithi023 to boot 6.3.0-rc3-g1776e8f494e9 2023-03-27T09:46:13.713 DEBUG:teuthology.orchestra.run.smithi023:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-03-27T09:46:16.678 INFO:teuthology.orchestra.run.smithi023.stderr:Generating grub configuration file ... 2023-03-27T09:46:18.318 INFO:teuthology.orchestra.run.smithi023.stderr:done 2023-03-27T09:46:18.320 DEBUG:teuthology.orchestra.run.smithi023:> mktemp 2023-03-27T09:46:18.338 INFO:teuthology.orchestra.run.smithi023.stdout:/tmp/tmp.TPssZ4FIVs 2023-03-27T09:46:18.339 DEBUG:teuthology.orchestra.run.smithi023:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.TPssZ4FIVs 2023-03-27T09:46:18.410 DEBUG:teuthology.orchestra.run.smithi023:> sudo chmod 0666 /tmp/tmp.TPssZ4FIVs 2023-03-27T09:46:18.561 DEBUG:teuthology.orchestra.remote:smithi023:/tmp/tmp.TPssZ4FIVs is 6KB 2023-03-27T09:46:18.612 DEBUG:teuthology.orchestra.run.smithi023:> rm -fr /tmp/tmp.TPssZ4FIVs 2023-03-27T09:46:18.628 DEBUG:teuthology.orchestra.run.smithi023:> sudo /bin/ls /boot/loader/entries || true 2023-03-27T09:46:18.693 INFO:teuthology.orchestra.run.smithi023.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-03-27T09:46:18.693 INFO:teuthology.orchestra.run.smithi023.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-03-27T09:46:18.693 INFO:teuthology.orchestra.run.smithi023.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.3.0-rc3-g1776e8f494e9.conf 2023-03-27T09:46:18.694 DEBUG:teuthology.orchestra.run.smithi023:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.3.0-rc3-g1776e8f494e9 2023-03-27T09:46:18.967 DEBUG:teuthology.orchestra.run.smithi023:> sudo shutdown -r now 2023-03-27T09:46:48.971 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-03-27T09:46:48.972 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi023.front.sepia.ceph.com' 2023-03-27T09:46:48.973 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-27T09:47:07.401 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.23 2023-03-27T09:47:37.403 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi023.front.sepia.ceph.com' 2023-03-27T09:47:37.404 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-27T09:47:40.485 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.23 2023-03-27T09:48:10.487 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi023.front.sepia.ceph.com' 2023-03-27T09:48:10.488 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-27T09:48:13.541 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.23 2023-03-27T09:48:43.542 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi023.front.sepia.ceph.com' 2023-03-27T09:48:43.543 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-27T09:48:43.897 DEBUG:teuthology.orchestra.run.smithi023:> true 2023-03-27T09:48:44.284 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi023.front.sepia.ceph.com' 2023-03-27T09:48:44.285 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.3.0-rc3-g1776e8f494e9"... 2023-03-27T09:48:44.285 DEBUG:teuthology.orchestra.run.smithi023:> uname -r 2023-03-27T09:48:44.324 INFO:teuthology.orchestra.run.smithi023.stdout:6.3.0-rc3-g1776e8f494e9 2023-03-27T09:48:44.325 DEBUG:teuthology.task.kernel:current kernel version is 6.3.0-rc3-g1776e8f494e9 vs 6.3.0-rc3-g1776e8f494e9 2023-03-27T09:48:44.325 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-03-27T09:48:44.325 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-03-27T09:48:45.328 DEBUG:teuthology.parallel:result is None 2023-03-27T09:48:45.329 INFO:teuthology.run_tasks:Running task internal.base... 2023-03-27T09:48:45.337 INFO:teuthology.task.internal:Creating test directory... 2023-03-27T09:48:45.337 DEBUG:teuthology.orchestra.run.smithi023:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-03-27T09:48:45.342 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-03-27T09:48:45.380 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-03-27T09:48:45.389 INFO:teuthology.run_tasks:Running task internal.archive... 2023-03-27T09:48:45.396 INFO:teuthology.task.internal:Creating archive directory... 2023-03-27T09:48:45.397 DEBUG:teuthology.orchestra.run.smithi023:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-03-27T09:48:45.429 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-03-27T09:48:45.485 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-03-27T09:48:45.493 INFO:teuthology.task.internal:Enabling coredump saving... 2023-03-27T09:48:45.494 DEBUG:teuthology.orchestra.run.smithi112:> test -f /run/.containerenv -o -f /.dockerenv 2023-03-27T09:48:45.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-27T09:48:45.513 DEBUG:teuthology.orchestra.run.smithi023:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-03-27T09:48:45.518 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-03-27T09:48:45.604 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-03-27T09:48:45.624 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-03-27T09:48:45.637 INFO:teuthology.orchestra.run.smithi023.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-03-27T09:48:45.667 INFO:teuthology.orchestra.run.smithi023.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-03-27T09:48:45.671 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-03-27T09:48:45.682 INFO:teuthology.task.internal:Configuring sudo... 2023-03-27T09:48:45.682 DEBUG:teuthology.orchestra.run.smithi023:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-03-27T09:48:45.686 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-03-27T09:48:45.744 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-03-27T09:48:45.754 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-03-27T09:48:45.755 DEBUG:teuthology.orchestra.run.smithi023:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-03-27T09:48:45.759 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-03-27T09:48:45.793 DEBUG:teuthology.orchestra.run.smithi023:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-03-27T09:48:45.916 DEBUG:teuthology.orchestra.run.smithi023:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-03-27T09:48:45.986 DEBUG:teuthology.orchestra.run.smithi023:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-03-27T09:48:46.058 DEBUG:teuthology.orchestra.run.smithi023:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-03-27T09:48:46.129 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2023-03-27T09:48:46.130 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-03-27T09:48:46.191 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-03-27T09:48:46.242 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-03-27T09:48:46.298 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-03-27T09:48:46.388 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-03-27T09:48:46.427 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-03-27T09:48:46.427 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-03-27T09:48:46.502 DEBUG:teuthology.orchestra.run.smithi023:> sudo service rsyslog restart 2023-03-27T09:48:46.510 DEBUG:teuthology.orchestra.run.smithi112:> sudo service rsyslog restart 2023-03-27T09:48:46.624 INFO:teuthology.orchestra.run.smithi023.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-03-27T09:48:46.635 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-03-27T09:48:47.075 INFO:teuthology.run_tasks:Running task internal.timer... 2023-03-27T09:48:47.096 INFO:teuthology.task.internal:Starting timer... 2023-03-27T09:48:47.097 INFO:teuthology.run_tasks:Running task pcp... 2023-03-27T09:48:47.154 INFO:teuthology.run_tasks:Running task selinux... 2023-03-27T09:48:47.165 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-03-27T09:48:47.166 DEBUG:teuthology.orchestra.run.smithi023:> sudo service auditd rotate 2023-03-27T09:48:47.268 INFO:teuthology.orchestra.run.smithi023.stdout:Rotating logs: 2023-03-27T09:48:47.273 DEBUG:teuthology.orchestra.run.smithi112:> sudo service auditd rotate 2023-03-27T09:48:47.351 INFO:teuthology.orchestra.run.smithi112.stdout:Rotating logs: 2023-03-27T09:48:47.353 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-03-27T09:48:47.354 DEBUG:teuthology.orchestra.run.smithi023:> /usr/sbin/getenforce 2023-03-27T09:48:47.397 INFO:teuthology.orchestra.run.smithi023.stdout:Permissive 2023-03-27T09:48:47.398 DEBUG:teuthology.orchestra.run.smithi112:> /usr/sbin/getenforce 2023-03-27T09:48:47.425 INFO:teuthology.orchestra.run.smithi112.stdout:Permissive 2023-03-27T09:48:47.425 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi023.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi112.front.sepia.ceph.com': 'permissive'} 2023-03-27T09:48:47.425 DEBUG:teuthology.orchestra.run.smithi023:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-03-27T09:48:47.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-27T09:48:47.535 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-03-27T09:48:47.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-27T09:48:47.565 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-03-27T09:48:47.566 DEBUG:teuthology.orchestra.run.smithi023:> sudo /usr/sbin/setenforce permissive 2023-03-27T09:48:47.627 DEBUG:teuthology.orchestra.run.smithi112:> sudo /usr/sbin/setenforce permissive 2023-03-27T09:48:47.660 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-03-27T09:48:47.672 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-03-27T09:48:47.673 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-03-27T09:48:47.695 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-03-27T09:48:47.697 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi023.front.sepia.ceph.com,smithi112.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-03-27T09:58:41.978 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi023.front.sepia.ceph.com'), Remote(name='ubuntu@smithi112.front.sepia.ceph.com')] 2023-03-27T09:58:41.980 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi023.front.sepia.ceph.com' 2023-03-27T09:58:41.981 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-27T09:58:42.066 DEBUG:teuthology.orchestra.run.smithi023:> true 2023-03-27T09:58:42.170 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2023-03-27T09:58:42.171 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-27T09:58:42.237 DEBUG:teuthology.orchestra.run.smithi112:> true 2023-03-27T09:58:42.262 INFO:teuthology.run_tasks:Running task clock... 2023-03-27T09:58:42.275 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-03-27T09:58:42.275 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-03-27T09:58:42.276 DEBUG:teuthology.orchestra.run.smithi023:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-03-27T09:58:42.280 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-03-27T09:58:42.280 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-03-27T09:58:42.333 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-03-27T09:58:42.343 INFO:teuthology.orchestra.run.smithi023.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-03-27T09:58:42.348 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-03-27T09:58:42.375 INFO:teuthology.orchestra.run.smithi023.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-03-27T09:58:42.419 INFO:teuthology.orchestra.run.smithi112.stderr:sudo: ntpd: command not found 2023-03-27T09:58:42.431 INFO:teuthology.orchestra.run.smithi112.stdout:506 Cannot talk to daemon 2023-03-27T09:58:42.445 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-03-27T09:58:42.459 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-03-27T09:58:42.471 INFO:teuthology.orchestra.run.smithi023.stderr:sudo: ntpd: command not found 2023-03-27T09:58:42.497 INFO:teuthology.orchestra.run.smithi023.stdout:506 Cannot talk to daemon 2023-03-27T09:58:42.528 INFO:teuthology.orchestra.run.smithi023.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-03-27T09:58:42.544 INFO:teuthology.orchestra.run.smithi112.stderr:bash: ntpq: command not found 2023-03-27T09:58:42.548 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-03-27T09:58:42.548 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2023-03-27T09:58:42.548 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-27T09:58:42.548 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-27T09:58:42.548 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-27T09:58:42.548 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-27T09:58:42.559 INFO:teuthology.orchestra.run.smithi023.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-03-27T09:58:42.647 INFO:teuthology.orchestra.run.smithi023.stderr:bash: ntpq: command not found 2023-03-27T09:58:42.654 INFO:teuthology.orchestra.run.smithi023.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-03-27T09:58:42.654 INFO:teuthology.orchestra.run.smithi023.stdout:=============================================================================== 2023-03-27T09:58:42.655 INFO:teuthology.orchestra.run.smithi023.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-27T09:58:42.655 INFO:teuthology.orchestra.run.smithi023.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-27T09:58:42.655 INFO:teuthology.orchestra.run.smithi023.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-27T09:58:42.655 INFO:teuthology.orchestra.run.smithi023.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-27T09:58:42.656 INFO:teuthology.run_tasks:Running task install... 2023-03-27T09:58:42.666 DEBUG:teuthology.task.install:project ceph 2023-03-27T09:58:42.667 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b70fdf973c2ffa23c0d8aa6918519da05d922302'}} 2023-03-27T09:58:42.667 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b70fdf973c2ffa23c0d8aa6918519da05d922302'} 2023-03-27T09:58:42.667 INFO:teuthology.task.install:Using flavor: default 2023-03-27T09:58:42.673 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-03-27T09:58:42.673 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-03-27T09:58:42.674 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b70fdf973c2ffa23c0d8aa6918519da05d922302', 'tag': None, 'wait_for_package': False} 2023-03-27T09:58:42.675 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b70fdf973c2ffa23c0d8aa6918519da05d922302 2023-03-27T09:58:42.677 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b70fdf973c2ffa23c0d8aa6918519da05d922302', 'tag': None, 'wait_for_package': False} 2023-03-27T09:58:42.677 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b70fdf973c2ffa23c0d8aa6918519da05d922302 2023-03-27T09:58:42.813 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-cephman-6/b70fdf973c2ffa23c0d8aa6918519da05d922302/centos/8/flavors/default/ 2023-03-27T09:58:42.814 INFO:teuthology.task.install.rpm:Package version is 18.0.0-2533.gb70fdf97 2023-03-27T09:58:42.818 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-cephman-6/b70fdf973c2ffa23c0d8aa6918519da05d922302/centos/8/flavors/default/ 2023-03-27T09:58:42.818 INFO:teuthology.task.install.rpm:Package version is 18.0.0-2533.gb70fdf97 2023-03-27T09:58:42.922 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-cephman-6/b70fdf973c2ffa23c0d8aa6918519da05d922302/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-cephman-6/b70fdf973c2ffa23c0d8aa6918519da05d922302/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-cephman-6/b70fdf973c2ffa23c0d8aa6918519da05d922302/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-03-27T09:58:42.922 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2023-03-27T09:58:42.923 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-03-27T09:58:42.933 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-cephman-6/b70fdf973c2ffa23c0d8aa6918519da05d922302/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-cephman-6/b70fdf973c2ffa23c0d8aa6918519da05d922302/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-cephman-6/b70fdf973c2ffa23c0d8aa6918519da05d922302/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-03-27T09:58:42.934 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-03-27T09:58:42.934 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-03-27T09:58:42.961 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-03-27T09:58:42.962 DEBUG:teuthology.orchestra.run.smithi112:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b70fdf973c2ffa23c0d8aa6918519da05d922302/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-03-27T09:58:42.984 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-03-27T09:58:42.984 DEBUG:teuthology.orchestra.run.smithi023:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b70fdf973c2ffa23c0d8aa6918519da05d922302/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-03-27T09:58:43.028 DEBUG:teuthology.orchestra.run.smithi112:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-03-27T09:58:43.087 DEBUG:teuthology.orchestra.run.smithi023:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-03-27T09:58:43.105 DEBUG:teuthology.orchestra.run.smithi112:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-03-27T09:58:43.171 INFO:teuthology.orchestra.run.smithi112.stdout:check_obsoletes = 1 2023-03-27T09:58:43.172 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum clean all 2023-03-27T09:58:43.173 DEBUG:teuthology.orchestra.run.smithi023:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-03-27T09:58:43.233 INFO:teuthology.orchestra.run.smithi023.stdout:check_obsoletes = 1 2023-03-27T09:58:43.235 DEBUG:teuthology.orchestra.run.smithi023:> sudo yum clean all 2023-03-27T09:58:43.541 INFO:teuthology.orchestra.run.smithi112.stdout:Updating Subscription Management repositories. 2023-03-27T09:58:43.696 INFO:teuthology.orchestra.run.smithi023.stdout:Updating Subscription Management repositories. 2023-03-27T09:58:43.800 INFO:teuthology.orchestra.run.smithi112.stdout:44 files removed 2023-03-27T09:58:43.846 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-03-27T09:58:43.987 INFO:teuthology.orchestra.run.smithi023.stdout:44 files removed 2023-03-27T09:58:44.042 DEBUG:teuthology.orchestra.run.smithi023:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-03-27T09:58:44.175 INFO:teuthology.orchestra.run.smithi112.stdout:Updating Subscription Management repositories. 2023-03-27T09:58:44.504 INFO:teuthology.orchestra.run.smithi023.stdout:Updating Subscription Management repositories. 2023-03-27T09:58:44.780 INFO:teuthology.orchestra.run.smithi112.stdout:ceph packages for x86_64 311 kB/s | 79 kB 00:00 2023-03-27T09:58:45.194 INFO:teuthology.orchestra.run.smithi023.stdout:ceph packages for x86_64 292 kB/s | 79 kB 00:00 2023-03-27T09:58:45.462 INFO:teuthology.orchestra.run.smithi023.stdout:ceph noarch packages 40 kB/s | 9.3 kB 00:00 2023-03-27T09:58:45.721 INFO:teuthology.orchestra.run.smithi023.stdout:ceph source packages 6.4 kB/s | 1.5 kB 00:00 2023-03-27T09:58:45.880 INFO:teuthology.orchestra.run.smithi023.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2023-03-27T09:58:46.265 INFO:teuthology.orchestra.run.smithi112.stdout:ceph noarch packages 6.4 kB/s | 9.3 kB 00:01 2023-03-27T09:58:46.524 INFO:teuthology.orchestra.run.smithi112.stdout:ceph source packages 6.3 kB/s | 1.5 kB 00:00 2023-03-27T09:58:46.675 INFO:teuthology.orchestra.run.smithi112.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2023-03-27T09:58:46.898 INFO:teuthology.orchestra.run.smithi023.stdout:Extra Packages for Enterprise Linux 14 MB/s | 14 MB 00:00 2023-03-27T09:58:47.734 INFO:teuthology.orchestra.run.smithi112.stdout:Extra Packages for Enterprise Linux 13 MB/s | 14 MB 00:01 2023-03-27T09:58:50.664 INFO:teuthology.orchestra.run.smithi023.stdout:lab-extras 573 kB/s | 24 kB 00:00 2023-03-27T09:58:50.859 INFO:teuthology.orchestra.run.smithi112.stdout:lab-extras 574 kB/s | 24 kB 00:00 2023-03-27T09:58:51.736 INFO:teuthology.orchestra.run.smithi112.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 59 MB/s | 47 MB 00:00 2023-03-27T09:58:51.850 INFO:teuthology.orchestra.run.smithi023.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 43 MB/s | 47 MB 00:01 2023-03-27T09:59:00.187 INFO:teuthology.orchestra.run.smithi112.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 59 MB/s | 53 MB 00:00 2023-03-27T09:59:00.967 INFO:teuthology.orchestra.run.smithi023.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 46 MB/s | 53 MB 00:01 2023-03-27T09:59:11.702 INFO:teuthology.orchestra.run.smithi112.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-03-27T09:59:11.702 INFO:teuthology.orchestra.run.smithi112.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-03-27T09:59:11.717 INFO:teuthology.orchestra.run.smithi112.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-03-27T09:59:11.718 INFO:teuthology.orchestra.run.smithi112.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-03-27T09:59:11.719 INFO:teuthology.orchestra.run.smithi112.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-03-27T09:59:11.720 INFO:teuthology.orchestra.run.smithi112.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-03-27T09:59:11.722 INFO:teuthology.orchestra.run.smithi112.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-03-27T09:59:11.725 INFO:teuthology.orchestra.run.smithi112.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-03-27T09:59:11.726 INFO:teuthology.orchestra.run.smithi112.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-03-27T09:59:11.856 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-03-27T09:59:11.868 INFO:teuthology.orchestra.run.smithi112.stdout:=========================================================================================================================== 2023-03-27T09:59:11.868 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repository Size 2023-03-27T09:59:11.868 INFO:teuthology.orchestra.run.smithi112.stdout:=========================================================================================================================== 2023-03-27T09:59:11.869 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-03-27T09:59:11.869 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-03-27T09:59:11.869 INFO:teuthology.orchestra.run.smithi112.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-03-27T09:59:11.869 INFO:teuthology.orchestra.run.smithi112.stdout: ceph x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 6.4 k 2023-03-27T09:59:11.869 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 4.9 M 2023-03-27T09:59:11.869 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 845 k 2023-03-27T09:59:11.870 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 165 k 2023-03-27T09:59:11.870 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 1.5 M 2023-03-27T09:59:11.870 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm noarch 2:18.0.0-2533.gb70fdf97.el8 ceph-noarch 120 k 2023-03-27T09:59:11.870 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard noarch 2:18.0.0-2533.gb70fdf97.el8 ceph-noarch 1.7 M 2023-03-27T09:59:11.870 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-2533.gb70fdf97.el8 ceph-noarch 7.4 M 2023-03-27T09:59:11.870 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook noarch 2:18.0.0-2533.gb70fdf97.el8 ceph-noarch 47 k 2023-03-27T09:59:11.870 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 13 M 2023-03-27T09:59:11.870 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 45 M 2023-03-27T09:59:11.871 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume noarch 2:18.0.0-2533.gb70fdf97.el8 ceph-noarch 263 k 2023-03-27T09:59:11.871 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm noarch 2:18.0.0-2533.gb70fdf97.el8 ceph-noarch 82 k 2023-03-27T09:59:11.871 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 258 k 2023-03-27T09:59:11.871 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-top noarch 2:18.0.0-2533.gb70fdf97.el8 ceph-noarch 18 k 2023-03-27T09:59:11.871 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-03-27T09:59:11.871 INFO:teuthology.orchestra.run.smithi112.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-03-27T09:59:11.871 INFO:teuthology.orchestra.run.smithi112.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-03-27T09:59:11.871 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 30 k 2023-03-27T09:59:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2 x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 768 k 2023-03-27T09:59:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 139 k 2023-03-27T09:59:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-03-27T09:59:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-03-27T09:59:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-03-27T09:59:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-03-27T09:59:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-03-27T09:59:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-03-27T09:59:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 190 k 2023-03-27T09:59:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 371 k 2023-03-27T09:59:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 365 k 2023-03-27T09:59:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 107 k 2023-03-27T09:59:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 95 k 2023-03-27T09:59:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 3.4 M 2023-03-27T09:59:11.874 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 180 k 2023-03-27T09:59:11.874 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-03-27T09:59:11.874 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrading: 2023-03-27T09:59:11.874 INFO:teuthology.orchestra.run.smithi112.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-03-27T09:59:11.874 INFO:teuthology.orchestra.run.smithi112.stdout: librados2 x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 3.6 M 2023-03-27T09:59:11.874 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1 x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 3.6 M 2023-03-27T09:59:11.874 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2023-03-27T09:59:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-03-27T09:59:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 21 M 2023-03-27T09:59:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards noarch 2:18.0.0-2533.gb70fdf97.el8 ceph-noarch 23 k 2023-03-27T09:59:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 2.3 M 2023-03-27T09:59:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core noarch 2:18.0.0-2533.gb70fdf97.el8 ceph-noarch 236 k 2023-03-27T09:59:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 3.9 M 2023-03-27T09:59:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 17 M 2023-03-27T09:59:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts noarch 2:18.0.0-2533.gb70fdf97.el8 ceph-noarch 15 k 2023-03-27T09:59:11.876 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 24 k 2023-03-27T09:59:11.876 INFO:teuthology.orchestra.run.smithi112.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-03-27T09:59:11.876 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-03-27T09:59:11.876 INFO:teuthology.orchestra.run.smithi112.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-03-27T09:59:11.876 INFO:teuthology.orchestra.run.smithi112.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-03-27T09:59:11.876 INFO:teuthology.orchestra.run.smithi112.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-03-27T09:59:11.876 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 137 k 2023-03-27T09:59:11.877 INFO:teuthology.orchestra.run.smithi112.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-03-27T09:59:11.877 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-03-27T09:59:11.877 INFO:teuthology.orchestra.run.smithi112.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-03-27T09:59:11.877 INFO:teuthology.orchestra.run.smithi112.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-03-27T09:59:11.877 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-03-27T09:59:11.877 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-03-27T09:59:11.877 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1 x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 474 k 2023-03-27T09:59:11.878 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-03-27T09:59:11.878 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2 x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 6.5 M 2023-03-27T09:59:11.878 INFO:teuthology.orchestra.run.smithi112.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-03-27T09:59:11.878 INFO:teuthology.orchestra.run.smithi112.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-03-27T09:59:11.878 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-03-27T09:59:11.878 INFO:teuthology.orchestra.run.smithi112.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-03-27T09:59:11.878 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-03-27T09:59:11.878 INFO:teuthology.orchestra.run.smithi112.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-03-27T09:59:11.879 INFO:teuthology.orchestra.run.smithi112.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-03-27T09:59:11.879 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-03-27T09:59:11.879 INFO:teuthology.orchestra.run.smithi112.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-03-27T09:59:11.879 INFO:teuthology.orchestra.run.smithi112.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-03-27T09:59:11.879 INFO:teuthology.orchestra.run.smithi112.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-03-27T09:59:11.879 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-03-27T09:59:11.879 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-03-27T09:59:11.880 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-03-27T09:59:11.880 INFO:teuthology.orchestra.run.smithi112.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-03-27T09:59:11.880 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-03-27T09:59:11.880 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 45 k 2023-03-27T09:59:11.880 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 117 k 2023-03-27T09:59:11.880 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-03-27T09:59:11.880 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-03-27T09:59:11.880 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-03-27T09:59:11.881 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-03-27T09:59:11.881 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-03-27T09:59:11.881 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-03-27T09:59:11.881 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-03-27T09:59:11.881 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-03-27T09:59:11.881 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-03-27T09:59:11.881 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-03-27T09:59:11.881 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-03-27T09:59:11.882 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-03-27T09:59:11.882 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-03-27T09:59:11.882 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-03-27T09:59:11.882 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-03-27T09:59:11.882 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-03-27T09:59:11.882 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-03-27T09:59:11.882 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-03-27T09:59:11.883 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-03-27T09:59:11.883 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-03-27T09:59:11.883 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-03-27T09:59:11.883 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-03-27T09:59:11.883 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-03-27T09:59:11.883 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-03-27T09:59:11.883 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-03-27T09:59:11.883 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-03-27T09:59:11.884 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-03-27T09:59:11.884 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-03-27T09:59:11.884 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-03-27T09:59:11.884 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-03-27T09:59:11.884 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-03-27T09:59:11.884 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-03-27T09:59:11.884 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-03-27T09:59:11.885 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-03-27T09:59:11.885 INFO:teuthology.orchestra.run.smithi112.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-03-27T09:59:11.885 INFO:teuthology.orchestra.run.smithi112.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-03-27T09:59:11.885 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-03-27T09:59:11.885 INFO:teuthology.orchestra.run.smithi112.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-03-27T09:59:11.885 INFO:teuthology.orchestra.run.smithi112.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-03-27T09:59:11.885 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-03-27T09:59:11.885 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-03-27T09:59:11.886 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-03-27T09:59:11.886 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-03-27T09:59:11.886 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-03-27T09:59:11.886 INFO:teuthology.orchestra.run.smithi112.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-03-27T09:59:11.886 INFO:teuthology.orchestra.run.smithi112.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-03-27T09:59:11.886 INFO:teuthology.orchestra.run.smithi112.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-03-27T09:59:11.886 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-03-27T09:59:11.886 INFO:teuthology.orchestra.run.smithi112.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-03-27T09:59:11.887 INFO:teuthology.orchestra.run.smithi112.stdout:Enabling module streams: 2023-03-27T09:59:11.887 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql 10 2023-03-27T09:59:11.887 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-03-27T09:59:11.887 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-03-27T09:59:11.887 INFO:teuthology.orchestra.run.smithi112.stdout:=========================================================================================================================== 2023-03-27T09:59:11.887 INFO:teuthology.orchestra.run.smithi112.stdout:Install 128 Packages 2023-03-27T09:59:11.887 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrade 3 Packages 2023-03-27T09:59:11.888 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-03-27T09:59:11.888 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 199 M 2023-03-27T09:59:11.888 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-03-27T09:59:12.004 INFO:teuthology.orchestra.run.smithi112.stdout:(1/131): ceph-18.0.0-2533.gb70fdf97.el8.x86_64. 53 kB/s | 6.4 kB 00:00 2023-03-27T09:59:12.696 INFO:teuthology.orchestra.run.smithi112.stdout:(2/131): ceph-fuse-18.0.0-2533.gb70fdf97.el8.x8 1.2 MB/s | 845 kB 00:00 2023-03-27T09:59:12.862 INFO:teuthology.orchestra.run.smithi112.stdout:(3/131): ceph-immutable-object-cache-18.0.0-253 995 kB/s | 165 kB 00:00 2023-03-27T09:59:13.254 INFO:teuthology.orchestra.run.smithi112.stdout:(4/131): ceph-base-18.0.0-2533.gb70fdf97.el8.x8 3.5 MB/s | 4.9 MB 00:01 2023-03-27T09:59:13.260 INFO:teuthology.orchestra.run.smithi023.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-03-27T09:59:13.261 INFO:teuthology.orchestra.run.smithi023.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-03-27T09:59:13.277 INFO:teuthology.orchestra.run.smithi023.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-03-27T09:59:13.278 INFO:teuthology.orchestra.run.smithi023.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-03-27T09:59:13.280 INFO:teuthology.orchestra.run.smithi023.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-03-27T09:59:13.281 INFO:teuthology.orchestra.run.smithi023.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-03-27T09:59:13.283 INFO:teuthology.orchestra.run.smithi023.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-03-27T09:59:13.287 INFO:teuthology.orchestra.run.smithi023.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-03-27T09:59:13.288 INFO:teuthology.orchestra.run.smithi023.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-03-27T09:59:13.437 INFO:teuthology.orchestra.run.smithi023.stdout:Dependencies resolved. 2023-03-27T09:59:13.451 INFO:teuthology.orchestra.run.smithi023.stdout:=========================================================================================================================== 2023-03-27T09:59:13.451 INFO:teuthology.orchestra.run.smithi023.stdout: Package Arch Version Repository Size 2023-03-27T09:59:13.451 INFO:teuthology.orchestra.run.smithi023.stdout:=========================================================================================================================== 2023-03-27T09:59:13.452 INFO:teuthology.orchestra.run.smithi023.stdout:Installing: 2023-03-27T09:59:13.452 INFO:teuthology.orchestra.run.smithi023.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-03-27T09:59:13.452 INFO:teuthology.orchestra.run.smithi023.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-03-27T09:59:13.452 INFO:teuthology.orchestra.run.smithi023.stdout: ceph x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 6.4 k 2023-03-27T09:59:13.453 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-base x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 4.9 M 2023-03-27T09:59:13.453 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-fuse x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 845 k 2023-03-27T09:59:13.453 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 165 k 2023-03-27T09:59:13.454 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-mgr x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 1.5 M 2023-03-27T09:59:13.454 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-mgr-cephadm noarch 2:18.0.0-2533.gb70fdf97.el8 ceph-noarch 120 k 2023-03-27T09:59:13.454 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-mgr-dashboard noarch 2:18.0.0-2533.gb70fdf97.el8 ceph-noarch 1.7 M 2023-03-27T09:59:13.454 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-2533.gb70fdf97.el8 ceph-noarch 7.4 M 2023-03-27T09:59:13.455 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-mgr-rook noarch 2:18.0.0-2533.gb70fdf97.el8 ceph-noarch 47 k 2023-03-27T09:59:13.455 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-radosgw x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 13 M 2023-03-27T09:59:13.455 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-test x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 45 M 2023-03-27T09:59:13.455 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-volume noarch 2:18.0.0-2533.gb70fdf97.el8 ceph-noarch 263 k 2023-03-27T09:59:13.456 INFO:teuthology.orchestra.run.smithi023.stdout: cephadm noarch 2:18.0.0-2533.gb70fdf97.el8 ceph-noarch 82 k 2023-03-27T09:59:13.456 INFO:teuthology.orchestra.run.smithi023.stdout: cephfs-mirror x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 258 k 2023-03-27T09:59:13.456 INFO:teuthology.orchestra.run.smithi023.stdout: cephfs-top noarch 2:18.0.0-2533.gb70fdf97.el8 ceph-noarch 18 k 2023-03-27T09:59:13.457 INFO:teuthology.orchestra.run.smithi023.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-03-27T09:59:13.457 INFO:teuthology.orchestra.run.smithi023.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-03-27T09:59:13.457 INFO:teuthology.orchestra.run.smithi023.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-03-27T09:59:13.457 INFO:teuthology.orchestra.run.smithi023.stdout: libcephfs-devel x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 30 k 2023-03-27T09:59:13.458 INFO:teuthology.orchestra.run.smithi023.stdout: libcephfs2 x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 768 k 2023-03-27T09:59:13.458 INFO:teuthology.orchestra.run.smithi023.stdout: librados-devel x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 139 k 2023-03-27T09:59:13.458 INFO:teuthology.orchestra.run.smithi023.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-03-27T09:59:13.458 INFO:teuthology.orchestra.run.smithi023.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-03-27T09:59:13.458 INFO:teuthology.orchestra.run.smithi023.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-03-27T09:59:13.458 INFO:teuthology.orchestra.run.smithi023.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-03-27T09:59:13.458 INFO:teuthology.orchestra.run.smithi023.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-03-27T09:59:13.459 INFO:teuthology.orchestra.run.smithi023.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-03-27T09:59:13.459 INFO:teuthology.orchestra.run.smithi023.stdout: python3-cephfs x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 190 k 2023-03-27T09:59:13.459 INFO:teuthology.orchestra.run.smithi023.stdout: python3-rados x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 371 k 2023-03-27T09:59:13.459 INFO:teuthology.orchestra.run.smithi023.stdout: python3-rbd x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 365 k 2023-03-27T09:59:13.460 INFO:teuthology.orchestra.run.smithi023.stdout: python3-rgw x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 107 k 2023-03-27T09:59:13.460 INFO:teuthology.orchestra.run.smithi023.stdout: rbd-fuse x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 95 k 2023-03-27T09:59:13.460 INFO:teuthology.orchestra.run.smithi023.stdout: rbd-mirror x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 3.4 M 2023-03-27T09:59:13.460 INFO:teuthology.orchestra.run.smithi023.stdout: rbd-nbd x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 180 k 2023-03-27T09:59:13.460 INFO:teuthology.orchestra.run.smithi023.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-03-27T09:59:13.461 INFO:teuthology.orchestra.run.smithi023.stdout:Upgrading: 2023-03-27T09:59:13.461 INFO:teuthology.orchestra.run.smithi023.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-03-27T09:59:13.461 INFO:teuthology.orchestra.run.smithi023.stdout: librados2 x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 3.6 M 2023-03-27T09:59:13.461 INFO:teuthology.orchestra.run.smithi023.stdout: librbd1 x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 3.6 M 2023-03-27T09:59:13.461 INFO:teuthology.orchestra.run.smithi023.stdout:Installing dependencies: 2023-03-27T09:59:13.461 INFO:teuthology.orchestra.run.smithi023.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-03-27T09:59:13.462 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-common x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 21 M 2023-03-27T09:59:13.462 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-grafana-dashboards noarch 2:18.0.0-2533.gb70fdf97.el8 ceph-noarch 23 k 2023-03-27T09:59:13.462 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-mds x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 2.3 M 2023-03-27T09:59:13.462 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-mgr-modules-core noarch 2:18.0.0-2533.gb70fdf97.el8 ceph-noarch 236 k 2023-03-27T09:59:13.462 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-mon x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 3.9 M 2023-03-27T09:59:13.462 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-osd x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 17 M 2023-03-27T09:59:13.462 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-prometheus-alerts noarch 2:18.0.0-2533.gb70fdf97.el8 ceph-noarch 15 k 2023-03-27T09:59:13.463 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-selinux x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 24 k 2023-03-27T09:59:13.463 INFO:teuthology.orchestra.run.smithi023.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-03-27T09:59:13.463 INFO:teuthology.orchestra.run.smithi023.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-03-27T09:59:13.463 INFO:teuthology.orchestra.run.smithi023.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-03-27T09:59:13.463 INFO:teuthology.orchestra.run.smithi023.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-03-27T09:59:13.464 INFO:teuthology.orchestra.run.smithi023.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-03-27T09:59:13.464 INFO:teuthology.orchestra.run.smithi023.stdout: libcephsqlite x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 137 k 2023-03-27T09:59:13.464 INFO:teuthology.orchestra.run.smithi023.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-03-27T09:59:13.464 INFO:teuthology.orchestra.run.smithi023.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-03-27T09:59:13.464 INFO:teuthology.orchestra.run.smithi023.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-03-27T09:59:13.464 INFO:teuthology.orchestra.run.smithi023.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-03-27T09:59:13.465 INFO:teuthology.orchestra.run.smithi023.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-03-27T09:59:13.465 INFO:teuthology.orchestra.run.smithi023.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-03-27T09:59:13.465 INFO:teuthology.orchestra.run.smithi023.stdout: libradosstriper1 x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 474 k 2023-03-27T09:59:13.465 INFO:teuthology.orchestra.run.smithi023.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-03-27T09:59:13.465 INFO:teuthology.orchestra.run.smithi023.stdout: librgw2 x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 6.5 M 2023-03-27T09:59:13.465 INFO:teuthology.orchestra.run.smithi023.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-03-27T09:59:13.465 INFO:teuthology.orchestra.run.smithi023.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-03-27T09:59:13.466 INFO:teuthology.orchestra.run.smithi023.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-03-27T09:59:13.466 INFO:teuthology.orchestra.run.smithi023.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-03-27T09:59:13.466 INFO:teuthology.orchestra.run.smithi023.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-03-27T09:59:13.466 INFO:teuthology.orchestra.run.smithi023.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-03-27T09:59:13.466 INFO:teuthology.orchestra.run.smithi023.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-03-27T09:59:13.466 INFO:teuthology.orchestra.run.smithi023.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-03-27T09:59:13.466 INFO:teuthology.orchestra.run.smithi023.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-03-27T09:59:13.467 INFO:teuthology.orchestra.run.smithi023.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-03-27T09:59:13.467 INFO:teuthology.orchestra.run.smithi023.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-03-27T09:59:13.467 INFO:teuthology.orchestra.run.smithi023.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-03-27T09:59:13.467 INFO:teuthology.orchestra.run.smithi023.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-03-27T09:59:13.467 INFO:teuthology.orchestra.run.smithi023.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-03-27T09:59:13.468 INFO:teuthology.orchestra.run.smithi023.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-03-27T09:59:13.468 INFO:teuthology.orchestra.run.smithi023.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-03-27T09:59:13.468 INFO:teuthology.orchestra.run.smithi023.stdout: python3-ceph-argparse x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 45 k 2023-03-27T09:59:13.468 INFO:teuthology.orchestra.run.smithi023.stdout: python3-ceph-common x86_64 2:18.0.0-2533.gb70fdf97.el8 ceph 117 k 2023-03-27T09:59:13.468 INFO:teuthology.orchestra.run.smithi023.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-03-27T09:59:13.468 INFO:teuthology.orchestra.run.smithi023.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-03-27T09:59:13.468 INFO:teuthology.orchestra.run.smithi023.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-03-27T09:59:13.468 INFO:teuthology.orchestra.run.smithi023.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-03-27T09:59:13.469 INFO:teuthology.orchestra.run.smithi023.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-03-27T09:59:13.469 INFO:teuthology.orchestra.run.smithi023.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-03-27T09:59:13.469 INFO:teuthology.orchestra.run.smithi023.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-03-27T09:59:13.469 INFO:teuthology.orchestra.run.smithi023.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-03-27T09:59:13.469 INFO:teuthology.orchestra.run.smithi023.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-03-27T09:59:13.469 INFO:teuthology.orchestra.run.smithi023.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-03-27T09:59:13.469 INFO:teuthology.orchestra.run.smithi023.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-03-27T09:59:13.470 INFO:teuthology.orchestra.run.smithi023.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-03-27T09:59:13.470 INFO:teuthology.orchestra.run.smithi023.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-03-27T09:59:13.470 INFO:teuthology.orchestra.run.smithi023.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-03-27T09:59:13.470 INFO:teuthology.orchestra.run.smithi023.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-03-27T09:59:13.470 INFO:teuthology.orchestra.run.smithi023.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-03-27T09:59:13.470 INFO:teuthology.orchestra.run.smithi023.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-03-27T09:59:13.470 INFO:teuthology.orchestra.run.smithi023.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-03-27T09:59:13.470 INFO:teuthology.orchestra.run.smithi023.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-03-27T09:59:13.471 INFO:teuthology.orchestra.run.smithi023.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-03-27T09:59:13.471 INFO:teuthology.orchestra.run.smithi023.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-03-27T09:59:13.471 INFO:teuthology.orchestra.run.smithi023.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-03-27T09:59:13.471 INFO:teuthology.orchestra.run.smithi023.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-03-27T09:59:13.471 INFO:teuthology.orchestra.run.smithi023.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-03-27T09:59:13.471 INFO:teuthology.orchestra.run.smithi023.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-03-27T09:59:13.472 INFO:teuthology.orchestra.run.smithi023.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-03-27T09:59:13.472 INFO:teuthology.orchestra.run.smithi023.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-03-27T09:59:13.472 INFO:teuthology.orchestra.run.smithi023.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-03-27T09:59:13.472 INFO:teuthology.orchestra.run.smithi023.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-03-27T09:59:13.472 INFO:teuthology.orchestra.run.smithi023.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-03-27T09:59:13.474 INFO:teuthology.orchestra.run.smithi023.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-03-27T09:59:13.474 INFO:teuthology.orchestra.run.smithi023.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-03-27T09:59:13.474 INFO:teuthology.orchestra.run.smithi023.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-03-27T09:59:13.475 INFO:teuthology.orchestra.run.smithi023.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-03-27T09:59:13.475 INFO:teuthology.orchestra.run.smithi023.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-03-27T09:59:13.475 INFO:teuthology.orchestra.run.smithi023.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-03-27T09:59:13.475 INFO:teuthology.orchestra.run.smithi023.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-03-27T09:59:13.476 INFO:teuthology.orchestra.run.smithi023.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-03-27T09:59:13.476 INFO:teuthology.orchestra.run.smithi023.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-03-27T09:59:13.476 INFO:teuthology.orchestra.run.smithi023.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-03-27T09:59:13.476 INFO:teuthology.orchestra.run.smithi023.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-03-27T09:59:13.476 INFO:teuthology.orchestra.run.smithi023.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-03-27T09:59:13.477 INFO:teuthology.orchestra.run.smithi023.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-03-27T09:59:13.477 INFO:teuthology.orchestra.run.smithi023.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-03-27T09:59:13.477 INFO:teuthology.orchestra.run.smithi023.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-03-27T09:59:13.477 INFO:teuthology.orchestra.run.smithi023.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-03-27T09:59:13.478 INFO:teuthology.orchestra.run.smithi023.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-03-27T09:59:13.478 INFO:teuthology.orchestra.run.smithi023.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-03-27T09:59:13.478 INFO:teuthology.orchestra.run.smithi023.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-03-27T09:59:13.478 INFO:teuthology.orchestra.run.smithi023.stdout:Enabling module streams: 2023-03-27T09:59:13.478 INFO:teuthology.orchestra.run.smithi023.stdout: postgresql 10 2023-03-27T09:59:13.478 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T09:59:13.479 INFO:teuthology.orchestra.run.smithi023.stdout:Transaction Summary 2023-03-27T09:59:13.479 INFO:teuthology.orchestra.run.smithi023.stdout:=========================================================================================================================== 2023-03-27T09:59:13.479 INFO:teuthology.orchestra.run.smithi023.stdout:Install 128 Packages 2023-03-27T09:59:13.479 INFO:teuthology.orchestra.run.smithi023.stdout:Upgrade 3 Packages 2023-03-27T09:59:13.479 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T09:59:13.480 INFO:teuthology.orchestra.run.smithi023.stdout:Total download size: 199 M 2023-03-27T09:59:13.480 INFO:teuthology.orchestra.run.smithi023.stdout:Downloading Packages: 2023-03-27T09:59:13.538 INFO:teuthology.orchestra.run.smithi112.stdout:(5/131): ceph-mgr-18.0.0-2533.gb70fdf97.el8.x86 5.5 MB/s | 1.5 MB 00:00 2023-03-27T09:59:13.617 INFO:teuthology.orchestra.run.smithi023.stdout:(1/131): ceph-18.0.0-2533.gb70fdf97.el8.x86_64. 45 kB/s | 6.4 kB 00:00 2023-03-27T09:59:13.788 INFO:teuthology.orchestra.run.smithi112.stdout:(6/131): ceph-mds-18.0.0-2533.gb70fdf97.el8.x86 2.5 MB/s | 2.3 MB 00:00 2023-03-27T09:59:13.905 INFO:teuthology.orchestra.run.smithi112.stdout:(7/131): ceph-mon-18.0.0-2533.gb70fdf97.el8.x86 11 MB/s | 3.9 MB 00:00 2023-03-27T09:59:14.009 INFO:teuthology.orchestra.run.smithi023.stdout:(2/131): ceph-fuse-18.0.0-2533.gb70fdf97.el8.x8 2.1 MB/s | 845 kB 00:00 2023-03-27T09:59:14.142 INFO:teuthology.orchestra.run.smithi023.stdout:(3/131): ceph-immutable-object-cache-18.0.0-253 1.2 MB/s | 165 kB 00:00 2023-03-27T09:59:15.131 INFO:teuthology.orchestra.run.smithi112.stdout:(8/131): ceph-radosgw-18.0.0-2533.gb70fdf97.el8 11 MB/s | 13 MB 00:01 2023-03-27T09:59:15.151 INFO:teuthology.orchestra.run.smithi023.stdout:(4/131): ceph-mds-18.0.0-2533.gb70fdf97.el8.x86 2.3 MB/s | 2.3 MB 00:01 2023-03-27T09:59:15.181 INFO:teuthology.orchestra.run.smithi112.stdout:(9/131): ceph-selinux-18.0.0-2533.gb70fdf97.el8 486 kB/s | 24 kB 00:00 2023-03-27T09:59:15.910 INFO:teuthology.orchestra.run.smithi023.stdout:(5/131): ceph-mgr-18.0.0-2533.gb70fdf97.el8.x86 2.0 MB/s | 1.5 MB 00:00 2023-03-27T09:59:16.244 INFO:teuthology.orchestra.run.smithi023.stdout:(6/131): ceph-base-18.0.0-2533.gb70fdf97.el8.x8 1.8 MB/s | 4.9 MB 00:02 2023-03-27T09:59:17.214 INFO:teuthology.orchestra.run.smithi112.stdout:(10/131): ceph-osd-18.0.0-2533.gb70fdf97.el8.x8 4.8 MB/s | 17 MB 00:03 2023-03-27T09:59:17.332 INFO:teuthology.orchestra.run.smithi112.stdout:(11/131): cephfs-mirror-18.0.0-2533.gb70fdf97.e 2.2 MB/s | 258 kB 00:00 2023-03-27T09:59:17.415 INFO:teuthology.orchestra.run.smithi112.stdout:(12/131): libcephfs-devel-18.0.0-2533.gb70fdf97 367 kB/s | 30 kB 00:00 2023-03-27T09:59:17.702 INFO:teuthology.orchestra.run.smithi023.stdout:(7/131): ceph-mon-18.0.0-2533.gb70fdf97.el8.x86 2.2 MB/s | 3.9 MB 00:01 2023-03-27T09:59:17.824 INFO:teuthology.orchestra.run.smithi112.stdout:(13/131): ceph-common-18.0.0-2533.gb70fdf97.el8 3.6 MB/s | 21 MB 00:05 2023-03-27T09:59:17.849 INFO:teuthology.orchestra.run.smithi112.stdout:(14/131): libcephfs2-18.0.0-2533.gb70fdf97.el8. 1.7 MB/s | 768 kB 00:00 2023-03-27T09:59:17.907 INFO:teuthology.orchestra.run.smithi112.stdout:(15/131): librados-devel-18.0.0-2533.gb70fdf97. 2.3 MB/s | 139 kB 00:00 2023-03-27T09:59:17.925 INFO:teuthology.orchestra.run.smithi112.stdout:(16/131): libcephsqlite-18.0.0-2533.gb70fdf97.e 1.3 MB/s | 137 kB 00:00 2023-03-27T09:59:18.075 INFO:teuthology.orchestra.run.smithi112.stdout:(17/131): libradosstriper1-18.0.0-2533.gb70fdf9 2.8 MB/s | 474 kB 00:00 2023-03-27T09:59:18.167 INFO:teuthology.orchestra.run.smithi112.stdout:(18/131): python3-ceph-argparse-18.0.0-2533.gb7 487 kB/s | 45 kB 00:00 2023-03-27T09:59:18.234 INFO:teuthology.orchestra.run.smithi112.stdout:(19/131): python3-ceph-common-18.0.0-2533.gb70f 1.7 MB/s | 117 kB 00:00 2023-03-27T09:59:18.334 INFO:teuthology.orchestra.run.smithi112.stdout:(20/131): python3-cephfs-18.0.0-2533.gb70fdf97. 1.9 MB/s | 190 kB 00:00 2023-03-27T09:59:18.501 INFO:teuthology.orchestra.run.smithi112.stdout:(21/131): python3-rados-18.0.0-2533.gb70fdf97.e 2.2 MB/s | 371 kB 00:00 2023-03-27T09:59:18.626 INFO:teuthology.orchestra.run.smithi112.stdout:(22/131): python3-rbd-18.0.0-2533.gb70fdf97.el8 2.8 MB/s | 365 kB 00:00 2023-03-27T09:59:19.110 INFO:teuthology.orchestra.run.smithi112.stdout:(23/131): ceph-test-18.0.0-2533.gb70fdf97.el8.x 11 MB/s | 45 MB 00:03 2023-03-27T09:59:19.135 INFO:teuthology.orchestra.run.smithi112.stdout:(24/131): python3-rgw-18.0.0-2533.gb70fdf97.el8 210 kB/s | 107 kB 00:00 2023-03-27T09:59:19.169 INFO:teuthology.orchestra.run.smithi112.stdout:(25/131): rbd-fuse-18.0.0-2533.gb70fdf97.el8.x8 1.6 MB/s | 95 kB 00:00 2023-03-27T09:59:19.236 INFO:teuthology.orchestra.run.smithi112.stdout:(26/131): rbd-nbd-18.0.0-2533.gb70fdf97.el8.x86 2.6 MB/s | 180 kB 00:00 2023-03-27T09:59:19.319 INFO:teuthology.orchestra.run.smithi112.stdout:(27/131): ceph-grafana-dashboards-18.0.0-2533.g 277 kB/s | 23 kB 00:00 2023-03-27T09:59:19.395 INFO:teuthology.orchestra.run.smithi112.stdout:(28/131): ceph-mgr-cephadm-18.0.0-2533.gb70fdf9 1.6 MB/s | 120 kB 00:00 2023-03-27T09:59:19.661 INFO:teuthology.orchestra.run.smithi112.stdout:(29/131): ceph-mgr-dashboard-18.0.0-2533.gb70fd 6.5 MB/s | 1.7 MB 00:00 2023-03-27T09:59:19.753 INFO:teuthology.orchestra.run.smithi112.stdout:(30/131): librgw2-18.0.0-2533.gb70fdf97.el8.x86 3.6 MB/s | 6.5 MB 00:01 2023-03-27T09:59:19.812 INFO:teuthology.orchestra.run.smithi112.stdout:(31/131): rbd-mirror-18.0.0-2533.gb70fdf97.el8. 5.1 MB/s | 3.4 MB 00:00 2023-03-27T09:59:19.846 INFO:teuthology.orchestra.run.smithi112.stdout:(32/131): ceph-mgr-modules-core-18.0.0-2533.gb7 2.5 MB/s | 236 kB 00:00 2023-03-27T09:59:19.888 INFO:teuthology.orchestra.run.smithi112.stdout:(33/131): ceph-mgr-rook-18.0.0-2533.gb70fdf97.e 630 kB/s | 47 kB 00:00 2023-03-27T09:59:19.921 INFO:teuthology.orchestra.run.smithi112.stdout:(34/131): ceph-prometheus-alerts-18.0.0-2533.gb 193 kB/s | 15 kB 00:00 2023-03-27T09:59:19.971 INFO:teuthology.orchestra.run.smithi112.stdout:(35/131): ceph-volume-18.0.0-2533.gb70fdf97.el8 3.1 MB/s | 263 kB 00:00 2023-03-27T09:59:20.048 INFO:teuthology.orchestra.run.smithi112.stdout:(36/131): cephadm-18.0.0-2533.gb70fdf97.el8.noa 651 kB/s | 82 kB 00:00 2023-03-27T09:59:20.072 INFO:teuthology.orchestra.run.smithi112.stdout:(37/131): cephfs-top-18.0.0-2533.gb70fdf97.el8. 179 kB/s | 18 kB 00:00 2023-03-27T09:59:20.256 INFO:teuthology.orchestra.run.smithi112.stdout:(38/131): ceph-mgr-diskprediction-local-18.0.0- 12 MB/s | 7.4 MB 00:00 2023-03-27T09:59:20.273 INFO:teuthology.orchestra.run.smithi112.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 294 kB/s | 59 kB 00:00 2023-03-27T09:59:20.332 INFO:teuthology.orchestra.run.smithi112.stdout:(40/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.1 MB/s | 306 kB 00:00 2023-03-27T09:59:20.418 INFO:teuthology.orchestra.run.smithi112.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 504 kB/s | 44 kB 00:00 2023-03-27T09:59:20.449 INFO:teuthology.orchestra.run.smithi112.stdout:(42/131): libunwind-1.3.1-3.el8.x86_64.rpm 390 kB/s | 75 kB 00:00 2023-03-27T09:59:20.482 INFO:teuthology.orchestra.run.smithi112.stdout:(43/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.2 MB/s | 476 kB 00:00 2023-03-27T09:59:20.508 INFO:teuthology.orchestra.run.smithi112.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 2.0 MB/s | 185 kB 00:00 2023-03-27T09:59:20.533 INFO:teuthology.orchestra.run.smithi112.stdout:(45/131): python3-cachetools-3.1.1-4.el8.noarch 393 kB/s | 33 kB 00:00 2023-03-27T09:59:20.558 INFO:teuthology.orchestra.run.smithi112.stdout:(46/131): python3-certifi-2018.10.15-7.el8.noar 215 kB/s | 16 kB 00:00 2023-03-27T09:59:20.584 INFO:teuthology.orchestra.run.smithi112.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.2 MB/s | 173 kB 00:00 2023-03-27T09:59:20.617 INFO:teuthology.orchestra.run.smithi112.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 1.4 MB/s | 81 kB 00:00 2023-03-27T09:59:20.642 INFO:teuthology.orchestra.run.smithi112.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 190 kB/s | 11 kB 00:00 2023-03-27T09:59:20.676 INFO:teuthology.orchestra.run.smithi112.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 300 kB/s | 18 kB 00:00 2023-03-27T09:59:20.701 INFO:teuthology.orchestra.run.smithi112.stdout:(51/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.2 MB/s | 384 kB 00:00 2023-03-27T09:59:20.735 INFO:teuthology.orchestra.run.smithi112.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 831 kB/s | 49 kB 00:00 2023-03-27T09:59:20.752 INFO:teuthology.orchestra.run.smithi023.stdout:(8/131): ceph-radosgw-18.0.0-2533.gb70fdf97.el8 4.3 MB/s | 13 MB 00:03 2023-03-27T09:59:20.760 INFO:teuthology.orchestra.run.smithi112.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 1.0 MB/s | 59 kB 00:00 2023-03-27T09:59:20.802 INFO:teuthology.orchestra.run.smithi023.stdout:(9/131): ceph-selinux-18.0.0-2533.gb70fdf97.el8 487 kB/s | 24 kB 00:00 2023-03-27T09:59:20.819 INFO:teuthology.orchestra.run.smithi112.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 721 kB/s | 60 kB 00:00 2023-03-27T09:59:20.852 INFO:teuthology.orchestra.run.smithi112.stdout:(55/131): python3-kubernetes-11.0.0-6.el8.noarc 5.6 MB/s | 1.2 MB 00:00 2023-03-27T09:59:20.878 INFO:teuthology.orchestra.run.smithi112.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 266 kB/s | 16 kB 00:00 2023-03-27T09:59:20.903 INFO:teuthology.orchestra.run.smithi112.stdout:(57/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.9 MB/s | 283 kB 00:00 2023-03-27T09:59:20.928 INFO:teuthology.orchestra.run.smithi112.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 445 kB/s | 34 kB 00:00 2023-03-27T09:59:20.954 INFO:teuthology.orchestra.run.smithi112.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 2.5 MB/s | 196 kB 00:00 2023-03-27T09:59:20.979 INFO:teuthology.orchestra.run.smithi112.stdout:(60/131): python3-rsa-4.8-1.el8.noarch.rpm 814 kB/s | 61 kB 00:00 2023-03-27T09:59:21.004 INFO:teuthology.orchestra.run.smithi112.stdout:(61/131): python3-simplegeneric-0.8.1-17.el8.no 248 kB/s | 19 kB 00:00 2023-03-27T09:59:21.029 INFO:teuthology.orchestra.run.smithi112.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 317 kB/s | 24 kB 00:00 2023-03-27T09:59:21.055 INFO:teuthology.orchestra.run.smithi112.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 387 kB/s | 29 kB 00:00 2023-03-27T09:59:21.080 INFO:teuthology.orchestra.run.smithi112.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 364 kB/s | 27 kB 00:00 2023-03-27T09:59:21.105 INFO:teuthology.orchestra.run.smithi112.stdout:(65/131): python3-waitress-1.2.1-2.el8.1.noarch 3.1 MB/s | 241 kB 00:00 2023-03-27T09:59:21.131 INFO:teuthology.orchestra.run.smithi112.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 3.2 MB/s | 251 kB 00:00 2023-03-27T09:59:21.169 INFO:teuthology.orchestra.run.smithi023.stdout:(10/131): ceph-osd-18.0.0-2533.gb70fdf97.el8.x8 3.4 MB/s | 17 MB 00:04 2023-03-27T09:59:21.181 INFO:teuthology.orchestra.run.smithi112.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 607 kB/s | 61 kB 00:00 2023-03-27T09:59:21.206 INFO:teuthology.orchestra.run.smithi112.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 304 kB/s | 23 kB 00:00 2023-03-27T09:59:21.231 INFO:teuthology.orchestra.run.smithi112.stdout:(69/131): python3-webtest-2.0.33-1.el8.noarch.r 678 kB/s | 85 kB 00:00 2023-03-27T09:59:21.257 INFO:teuthology.orchestra.run.smithi112.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2023-03-27T09:59:21.316 INFO:teuthology.orchestra.run.smithi112.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.0 MB/s | 110 kB 00:00 2023-03-27T09:59:21.366 INFO:teuthology.orchestra.run.smithi112.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-03-27T09:59:21.395 INFO:teuthology.orchestra.run.smithi023.stdout:(11/131): ceph-common-18.0.0-2533.gb70fdf97.el8 2.7 MB/s | 21 MB 00:07 2023-03-27T09:59:21.408 INFO:teuthology.orchestra.run.smithi112.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-03-27T09:59:21.419 INFO:teuthology.orchestra.run.smithi023.stdout:(12/131): cephfs-mirror-18.0.0-2533.gb70fdf97.e 1.0 MB/s | 258 kB 00:00 2023-03-27T09:59:21.474 INFO:teuthology.orchestra.run.smithi023.stdout:(13/131): libcephfs-devel-18.0.0-2533.gb70fdf97 386 kB/s | 30 kB 00:00 2023-03-27T09:59:21.475 INFO:teuthology.orchestra.run.smithi112.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 4.7 MB/s | 323 kB 00:00 2023-03-27T09:59:21.538 INFO:teuthology.orchestra.run.smithi112.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 616 kB/s | 39 kB 00:00 2023-03-27T09:59:21.545 INFO:teuthology.orchestra.run.smithi023.stdout:(14/131): libcephsqlite-18.0.0-2533.gb70fdf97.e 1.9 MB/s | 137 kB 00:00 2023-03-27T09:59:21.587 INFO:teuthology.orchestra.run.smithi023.stdout:(15/131): libcephfs2-18.0.0-2533.gb70fdf97.el8. 4.5 MB/s | 768 kB 00:00 2023-03-27T09:59:21.592 INFO:teuthology.orchestra.run.smithi112.stdout:(76/131): bison-3.0.4-10.el8.x86_64.rpm 13 MB/s | 688 kB 00:00 2023-03-27T09:59:21.612 INFO:teuthology.orchestra.run.smithi023.stdout:(16/131): librados-devel-18.0.0-2533.gb70fdf97. 2.0 MB/s | 139 kB 00:00 2023-03-27T09:59:21.650 INFO:teuthology.orchestra.run.smithi112.stdout:(77/131): python3-jinja2-2.10.1-3.el8.noarch.rp 9.0 MB/s | 538 kB 00:00 2023-03-27T09:59:21.679 INFO:teuthology.orchestra.run.smithi023.stdout:(17/131): libradosstriper1-18.0.0-2533.gb70fdf9 5.1 MB/s | 474 kB 00:00 2023-03-27T09:59:21.684 INFO:teuthology.orchestra.run.smithi112.stdout:(78/131): python3-mako-1.0.6-13.el8.noarch.rpm 4.6 MB/s | 157 kB 00:00 2023-03-27T09:59:21.729 INFO:teuthology.orchestra.run.smithi023.stdout:(18/131): python3-ceph-argparse-18.0.0-2533.gb7 897 kB/s | 45 kB 00:00 2023-03-27T09:59:21.780 INFO:teuthology.orchestra.run.smithi023.stdout:(19/131): python3-ceph-common-18.0.0-2533.gb70f 2.3 MB/s | 117 kB 00:00 2023-03-27T09:59:21.826 INFO:teuthology.orchestra.run.smithi112.stdout:(79/131): python3-scipy-1.0.0-21.module+el8.5.0 23 MB/s | 14 MB 00:00 2023-03-27T09:59:21.839 INFO:teuthology.orchestra.run.smithi023.stdout:(20/131): python3-cephfs-18.0.0-2533.gb70fdf97. 3.2 MB/s | 190 kB 00:00 2023-03-27T09:59:21.868 INFO:teuthology.orchestra.run.smithi112.stdout:(80/131): thrift-0.13.0-2.el8.x86_64.rpm 2.5 MB/s | 1.7 MB 00:00 2023-03-27T09:59:21.919 INFO:teuthology.orchestra.run.smithi112.stdout:(81/131): oniguruma-6.8.2-2.el8.x86_64.rpm 800 kB/s | 187 kB 00:00 2023-03-27T09:59:21.922 INFO:teuthology.orchestra.run.smithi023.stdout:(21/131): python3-rados-18.0.0-2533.gb70fdf97.e 4.4 MB/s | 371 kB 00:00 2023-03-27T09:59:21.968 INFO:teuthology.orchestra.run.smithi112.stdout:(82/131): automake-1.16.1-7.el8.noarch.rpm 4.9 MB/s | 713 kB 00:00 2023-03-27T09:59:21.997 INFO:teuthology.orchestra.run.smithi023.stdout:(22/131): python3-rbd-18.0.0-2533.gb70fdf97.el8 4.8 MB/s | 365 kB 00:00 2023-03-27T09:59:22.048 INFO:teuthology.orchestra.run.smithi023.stdout:(23/131): python3-rgw-18.0.0-2533.gb70fdf97.el8 2.1 MB/s | 107 kB 00:00 2023-03-27T09:59:22.098 INFO:teuthology.orchestra.run.smithi023.stdout:(24/131): rbd-fuse-18.0.0-2533.gb70fdf97.el8.x8 1.9 MB/s | 95 kB 00:00 2023-03-27T09:59:22.194 INFO:teuthology.orchestra.run.smithi112.stdout:(83/131): python3-jsonpatch-1.21-2.el8.noarch.r 99 kB/s | 27 kB 00:00 2023-03-27T09:59:22.228 INFO:teuthology.orchestra.run.smithi112.stdout:(84/131): flex-2.6.1-9.el8.x86_64.rpm 1.2 MB/s | 320 kB 00:00 2023-03-27T09:59:22.253 INFO:teuthology.orchestra.run.smithi112.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 751 kB/s | 44 kB 00:00 2023-03-27T09:59:22.303 INFO:teuthology.orchestra.run.smithi112.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 9.0 MB/s | 457 kB 00:00 2023-03-27T09:59:22.345 INFO:teuthology.orchestra.run.smithi112.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 5.3 MB/s | 225 kB 00:00 2023-03-27T09:59:22.387 INFO:teuthology.orchestra.run.smithi112.stdout:(88/131): uuid-1.6.2-43.el8.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2023-03-27T09:59:22.454 INFO:teuthology.orchestra.run.smithi112.stdout:(89/131): python3-numpy-1.14.3-10.el8.x86_64.rp 17 MB/s | 3.7 MB 00:00 2023-03-27T09:59:22.479 INFO:teuthology.orchestra.run.smithi112.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.3 MB/s | 126 kB 00:00 2023-03-27T09:59:22.504 INFO:teuthology.orchestra.run.smithi112.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 399 kB/s | 20 kB 00:00 2023-03-27T09:59:22.523 INFO:teuthology.orchestra.run.smithi023.stdout:(25/131): librgw2-18.0.0-2533.gb70fdf97.el8.x86 7.1 MB/s | 6.5 MB 00:00 2023-03-27T09:59:22.530 INFO:teuthology.orchestra.run.smithi112.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-03-27T09:59:22.605 INFO:teuthology.orchestra.run.smithi112.stdout:(93/131): python3-babel-2.5.1-7.el8.noarch.rpm 6.5 MB/s | 4.8 MB 00:00 2023-03-27T09:59:22.607 INFO:teuthology.orchestra.run.smithi023.stdout:(26/131): rbd-mirror-18.0.0-2533.gb70fdf97.el8. 6.8 MB/s | 3.4 MB 00:00 2023-03-27T09:59:22.632 INFO:teuthology.orchestra.run.smithi023.stdout:(27/131): rbd-nbd-18.0.0-2533.gb70fdf97.el8.x86 1.6 MB/s | 180 kB 00:00 2023-03-27T09:59:22.658 INFO:teuthology.orchestra.run.smithi023.stdout:(28/131): ceph-grafana-dashboards-18.0.0-2533.g 459 kB/s | 23 kB 00:00 2023-03-27T09:59:22.666 INFO:teuthology.orchestra.run.smithi112.stdout:(94/131): libtool-2.4.6-25.el8.x86_64.rpm 4.3 MB/s | 709 kB 00:00 2023-03-27T09:59:22.683 INFO:teuthology.orchestra.run.smithi023.stdout:(29/131): ceph-mgr-cephadm-18.0.0-2533.gb70fdf9 2.4 MB/s | 120 kB 00:00 2023-03-27T09:59:22.706 INFO:teuthology.orchestra.run.smithi112.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 1.9 MB/s | 198 kB 00:00 2023-03-27T09:59:22.748 INFO:teuthology.orchestra.run.smithi112.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-03-27T09:59:22.790 INFO:teuthology.orchestra.run.smithi112.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-03-27T09:59:22.882 INFO:teuthology.orchestra.run.smithi112.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 14 MB/s | 4.9 MB 00:00 2023-03-27T09:59:22.908 INFO:teuthology.orchestra.run.smithi023.stdout:(30/131): ceph-mgr-dashboard-18.0.0-2533.gb70fd 6.9 MB/s | 1.7 MB 00:00 2023-03-27T09:59:22.948 INFO:teuthology.orchestra.run.smithi112.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 17 MB/s | 4.7 MB 00:00 2023-03-27T09:59:22.992 INFO:teuthology.orchestra.run.smithi023.stdout:(31/131): ceph-mgr-modules-core-18.0.0-2533.gb7 2.8 MB/s | 236 kB 00:00 2023-03-27T09:59:22.999 INFO:teuthology.orchestra.run.smithi112.stdout:(100/131): postgresql-contrib-10.21-2.module+el 6.7 MB/s | 808 kB 00:00 2023-03-27T09:59:23.042 INFO:teuthology.orchestra.run.smithi023.stdout:(32/131): ceph-mgr-rook-18.0.0-2533.gb70fdf97.e 953 kB/s | 47 kB 00:00 2023-03-27T09:59:23.074 INFO:teuthology.orchestra.run.smithi112.stdout:(101/131): postgresql-10.21-2.module+el8.6.0+15 12 MB/s | 1.5 MB 00:00 2023-03-27T09:59:23.099 INFO:teuthology.orchestra.run.smithi112.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 468 kB/s | 47 kB 00:00 2023-03-27T09:59:23.101 INFO:teuthology.orchestra.run.smithi023.stdout:(33/131): ceph-prometheus-alerts-18.0.0-2533.gb 248 kB/s | 15 kB 00:00 2023-03-27T09:59:23.125 INFO:teuthology.orchestra.run.smithi112.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 3.9 MB/s | 200 kB 00:00 2023-03-27T09:59:23.150 INFO:teuthology.orchestra.run.smithi112.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 848 kB/s | 43 kB 00:00 2023-03-27T09:59:23.175 INFO:teuthology.orchestra.run.smithi112.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-03-27T09:59:23.176 INFO:teuthology.orchestra.run.smithi023.stdout:(34/131): ceph-volume-18.0.0-2533.gb70fdf97.el8 3.4 MB/s | 263 kB 00:00 2023-03-27T09:59:23.201 INFO:teuthology.orchestra.run.smithi112.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 959 kB/s | 48 kB 00:00 2023-03-27T09:59:23.226 INFO:teuthology.orchestra.run.smithi112.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-03-27T09:59:23.226 INFO:teuthology.orchestra.run.smithi023.stdout:(35/131): cephadm-18.0.0-2533.gb70fdf97.el8.noa 1.6 MB/s | 82 kB 00:00 2023-03-27T09:59:23.252 INFO:teuthology.orchestra.run.smithi112.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 849 kB/s | 43 kB 00:00 2023-03-27T09:59:23.268 INFO:teuthology.orchestra.run.smithi023.stdout:(36/131): cephfs-top-18.0.0-2533.gb70fdf97.el8. 436 kB/s | 18 kB 00:00 2023-03-27T09:59:23.276 INFO:teuthology.orchestra.run.smithi112.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-03-27T09:59:23.302 INFO:teuthology.orchestra.run.smithi112.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.5 MB/s | 229 kB 00:00 2023-03-27T09:59:23.369 INFO:teuthology.orchestra.run.smithi112.stdout:(111/131): postgresql-server-10.21-2.module+el8 8.8 MB/s | 5.1 MB 00:00 2023-03-27T09:59:23.419 INFO:teuthology.orchestra.run.smithi112.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 160 kB/s | 19 kB 00:00 2023-03-27T09:59:23.444 INFO:teuthology.orchestra.run.smithi112.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 2.0 MB/s | 155 kB 00:00 2023-03-27T09:59:23.469 INFO:teuthology.orchestra.run.smithi112.stdout:(114/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 2.8 MB/s | 560 kB 00:00 2023-03-27T09:59:23.486 INFO:teuthology.orchestra.run.smithi112.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 428 kB/s | 18 kB 00:00 2023-03-27T09:59:23.520 INFO:teuthology.orchestra.run.smithi112.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 5.4 MB/s | 559 kB 00:00 2023-03-27T09:59:23.545 INFO:teuthology.orchestra.run.smithi112.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-03-27T09:59:23.579 INFO:teuthology.orchestra.run.smithi112.stdout:(118/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.4 MB/s | 605 kB 00:00 2023-03-27T09:59:23.602 INFO:teuthology.orchestra.run.smithi023.stdout:(37/131): ceph-mgr-diskprediction-local-18.0.0- 8.1 MB/s | 7.4 MB 00:00 2023-03-27T09:59:23.604 INFO:teuthology.orchestra.run.smithi112.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.2 MB/s | 72 kB 00:00 2023-03-27T09:59:23.627 INFO:teuthology.orchestra.run.smithi023.stdout:(38/131): gperftools-libs-2.7-9.el8.x86_64.rpm 854 kB/s | 306 kB 00:00 2023-03-27T09:59:23.629 INFO:teuthology.orchestra.run.smithi112.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-03-27T09:59:23.655 INFO:teuthology.orchestra.run.smithi112.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-03-27T09:59:23.680 INFO:teuthology.orchestra.run.smithi112.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-03-27T09:59:23.703 INFO:teuthology.orchestra.run.smithi023.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 998 kB/s | 75 kB 00:00 2023-03-27T09:59:23.706 INFO:teuthology.orchestra.run.smithi112.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-03-27T09:59:23.731 INFO:teuthology.orchestra.run.smithi112.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 763 kB/s | 39 kB 00:00 2023-03-27T09:59:23.798 INFO:teuthology.orchestra.run.smithi112.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 6.8 MB/s | 644 kB 00:00 2023-03-27T09:59:23.820 INFO:teuthology.orchestra.run.smithi023.stdout:(40/131): liboath-2.6.2-3.el8.x86_64.rpm 271 kB/s | 59 kB 00:00 2023-03-27T09:59:23.823 INFO:teuthology.orchestra.run.smithi112.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 1.8 MB/s | 171 kB 00:00 2023-03-27T09:59:23.845 INFO:teuthology.orchestra.run.smithi023.stdout:(41/131): python3-asyncssh-2.7.0-2.el8.noarch.r 3.3 MB/s | 476 kB 00:00 2023-03-27T09:59:23.848 INFO:teuthology.orchestra.run.smithi112.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-03-27T09:59:23.887 INFO:teuthology.orchestra.run.smithi023.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 654 kB/s | 44 kB 00:00 2023-03-27T09:59:23.907 INFO:teuthology.orchestra.run.smithi112.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.0 MB/s | 2.3 MB 00:00 2023-03-27T09:59:23.912 INFO:teuthology.orchestra.run.smithi023.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 2.7 MB/s | 185 kB 00:00 2023-03-27T09:59:23.954 INFO:teuthology.orchestra.run.smithi023.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 494 kB/s | 33 kB 00:00 2023-03-27T09:59:23.966 INFO:teuthology.orchestra.run.smithi112.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-03-27T09:59:23.979 INFO:teuthology.orchestra.run.smithi023.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 244 kB/s | 16 kB 00:00 2023-03-27T09:59:24.088 INFO:teuthology.orchestra.run.smithi023.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 3.5 MB/s | 384 kB 00:00 2023-03-27T09:59:24.113 INFO:teuthology.orchestra.run.smithi023.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.1 MB/s | 173 kB 00:00 2023-03-27T09:59:24.155 INFO:teuthology.orchestra.run.smithi023.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 1.2 MB/s | 81 kB 00:00 2023-03-27T09:59:24.180 INFO:teuthology.orchestra.run.smithi023.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 166 kB/s | 11 kB 00:00 2023-03-27T09:59:24.222 INFO:teuthology.orchestra.run.smithi023.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 264 kB/s | 18 kB 00:00 2023-03-27T09:59:24.224 INFO:teuthology.orchestra.run.smithi112.stdout:(130/131): librbd1-18.0.0-2533.gb70fdf97.el8.x8 9.5 MB/s | 3.6 MB 00:00 2023-03-27T09:59:24.289 INFO:teuthology.orchestra.run.smithi023.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 733 kB/s | 49 kB 00:00 2023-03-27T09:59:24.356 INFO:teuthology.orchestra.run.smithi023.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 882 kB/s | 59 kB 00:00 2023-03-27T09:59:24.366 INFO:teuthology.orchestra.run.smithi112.stdout:(131/131): librados2-18.0.0-2533.gb70fdf97.el8. 6.7 MB/s | 3.6 MB 00:00 2023-03-27T09:59:24.369 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-03-27T09:59:24.369 INFO:teuthology.orchestra.run.smithi112.stdout:Total 16 MB/s | 199 MB 00:12 2023-03-27T09:59:24.423 INFO:teuthology.orchestra.run.smithi023.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 907 kB/s | 60 kB 00:00 2023-03-27T09:59:24.490 INFO:teuthology.orchestra.run.smithi023.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 4.1 MB/s | 283 kB 00:00 2023-03-27T09:59:24.549 INFO:teuthology.orchestra.run.smithi023.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 268 kB/s | 16 kB 00:00 2023-03-27T09:59:24.607 INFO:teuthology.orchestra.run.smithi023.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 579 kB/s | 34 kB 00:00 2023-03-27T09:59:24.674 INFO:teuthology.orchestra.run.smithi023.stdout:(57/131): python3-routes-2.4.1-12.el8.noarch.rp 2.9 MB/s | 196 kB 00:00 2023-03-27T09:59:24.716 INFO:teuthology.orchestra.run.smithi023.stdout:(58/131): python3-kubernetes-11.0.0-6.el8.noarc 2.2 MB/s | 1.2 MB 00:00 2023-03-27T09:59:24.742 INFO:teuthology.orchestra.run.smithi023.stdout:(59/131): python3-rsa-4.8-1.el8.noarch.rpm 923 kB/s | 61 kB 00:00 2023-03-27T09:59:24.910 INFO:teuthology.orchestra.run.smithi023.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 97 kB/s | 19 kB 00:00 2023-03-27T09:59:24.943 INFO:teuthology.orchestra.run.smithi023.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 119 kB/s | 24 kB 00:00 2023-03-27T09:59:24.976 INFO:teuthology.orchestra.run.smithi023.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 442 kB/s | 29 kB 00:00 2023-03-27T09:59:25.010 INFO:teuthology.orchestra.run.smithi023.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 412 kB/s | 27 kB 00:00 2023-03-27T09:59:25.077 INFO:teuthology.orchestra.run.smithi023.stdout:(64/131): python3-webob-1.8.5-1.el8.1.noarch.rp 3.7 MB/s | 251 kB 00:00 2023-03-27T09:59:25.119 INFO:teuthology.orchestra.run.smithi023.stdout:(65/131): python3-waitress-1.2.1-2.el8.1.noarch 1.7 MB/s | 241 kB 00:00 2023-03-27T09:59:25.144 INFO:teuthology.orchestra.run.smithi023.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 920 kB/s | 61 kB 00:00 2023-03-27T09:59:25.185 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-03-27T09:59:25.186 INFO:teuthology.orchestra.run.smithi023.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.2 MB/s | 85 kB 00:00 2023-03-27T09:59:25.211 INFO:teuthology.orchestra.run.smithi023.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 343 kB/s | 23 kB 00:00 2023-03-27T09:59:25.237 INFO:teuthology.orchestra.run.smithi023.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-03-27T09:59:25.304 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-03-27T09:59:25.304 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-03-27T09:59:25.687 INFO:teuthology.orchestra.run.smithi023.stdout:(70/131): python3-scipy-1.0.0-21.module+el8.5.0 30 MB/s | 14 MB 00:00 2023-03-27T09:59:25.729 INFO:teuthology.orchestra.run.smithi023.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-03-27T09:59:25.771 INFO:teuthology.orchestra.run.smithi023.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-03-27T09:59:25.821 INFO:teuthology.orchestra.run.smithi023.stdout:(73/131): thrift-0.13.0-2.el8.x86_64.rpm 2.7 MB/s | 1.7 MB 00:00 2023-03-27T09:59:25.855 INFO:teuthology.orchestra.run.smithi023.stdout:(74/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.0 MB/s | 259 kB 00:00 2023-03-27T09:59:26.280 INFO:teuthology.orchestra.run.smithi023.stdout:(75/131): ceph-test-18.0.0-2533.gb70fdf97.el8.x 8.2 MB/s | 45 MB 00:05 2023-03-27T09:59:26.306 INFO:teuthology.orchestra.run.smithi023.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 88 kB/s | 39 kB 00:00 2023-03-27T09:59:26.347 INFO:teuthology.orchestra.run.smithi023.stdout:(77/131): socat-1.7.4.1-1.el8.x86_64.rpm 615 kB/s | 323 kB 00:00 2023-03-27T09:59:26.398 INFO:teuthology.orchestra.run.smithi023.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 6.3 MB/s | 538 kB 00:00 2023-03-27T09:59:26.400 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-03-27T09:59:26.404 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-03-27T09:59:26.431 INFO:teuthology.orchestra.run.smithi023.stdout:(79/131): bison-3.0.4-10.el8.x86_64.rpm 4.5 MB/s | 688 kB 00:00 2023-03-27T09:59:26.457 INFO:teuthology.orchestra.run.smithi023.stdout:(80/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.4 MB/s | 157 kB 00:00 2023-03-27T09:59:26.482 INFO:teuthology.orchestra.run.smithi023.stdout:(81/131): oniguruma-6.8.2-2.el8.x86_64.rpm 2.2 MB/s | 187 kB 00:00 2023-03-27T09:59:26.532 INFO:teuthology.orchestra.run.smithi023.stdout:(82/131): automake-1.16.1-7.el8.noarch.rpm 6.9 MB/s | 713 kB 00:00 2023-03-27T09:59:26.557 INFO:teuthology.orchestra.run.smithi023.stdout:(83/131): python3-jsonpatch-1.21-2.el8.noarch.r 352 kB/s | 27 kB 00:00 2023-03-27T09:59:26.591 INFO:teuthology.orchestra.run.smithi023.stdout:(84/131): flex-2.6.1-9.el8.x86_64.rpm 5.3 MB/s | 320 kB 00:00 2023-03-27T09:59:26.616 INFO:teuthology.orchestra.run.smithi023.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 752 kB/s | 44 kB 00:00 2023-03-27T09:59:26.675 INFO:teuthology.orchestra.run.smithi023.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 7.7 MB/s | 457 kB 00:00 2023-03-27T09:59:26.717 INFO:teuthology.orchestra.run.smithi023.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 5.3 MB/s | 225 kB 00:00 2023-03-27T09:59:26.768 INFO:teuthology.orchestra.run.smithi023.stdout:(88/131): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-03-27T09:59:26.818 INFO:teuthology.orchestra.run.smithi023.stdout:(89/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-03-27T09:59:26.860 INFO:teuthology.orchestra.run.smithi023.stdout:(90/131): python3-jsonpointer-1.10-11.el8.noarc 481 kB/s | 20 kB 00:00 2023-03-27T09:59:26.927 INFO:teuthology.orchestra.run.smithi023.stdout:(91/131): python3-numpy-1.14.3-10.el8.x86_64.rp 11 MB/s | 3.7 MB 00:00 2023-03-27T09:59:26.952 INFO:teuthology.orchestra.run.smithi023.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.1 MB/s | 103 kB 00:00 2023-03-27T09:59:27.019 INFO:teuthology.orchestra.run.smithi023.stdout:(93/131): python3-babel-2.5.1-7.el8.noarch.rpm 8.5 MB/s | 4.8 MB 00:00 2023-03-27T09:59:27.061 INFO:teuthology.orchestra.run.smithi023.stdout:(94/131): libtool-2.4.6-25.el8.x86_64.rpm 5.2 MB/s | 709 kB 00:00 2023-03-27T09:59:27.086 INFO:teuthology.orchestra.run.smithi023.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 2.9 MB/s | 198 kB 00:00 2023-03-27T09:59:27.128 INFO:teuthology.orchestra.run.smithi023.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-03-27T09:59:27.178 INFO:teuthology.orchestra.run.smithi023.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 7.0 MB/s | 354 kB 00:00 2023-03-27T09:59:27.346 INFO:teuthology.orchestra.run.smithi023.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 12 MB/s | 4.9 MB 00:00 2023-03-27T09:59:27.429 INFO:teuthology.orchestra.run.smithi023.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 13 MB/s | 4.7 MB 00:00 2023-03-27T09:59:27.479 INFO:teuthology.orchestra.run.smithi023.stdout:(100/131): postgresql-contrib-10.21-2.module+el 5.9 MB/s | 808 kB 00:00 2023-03-27T09:59:27.513 INFO:teuthology.orchestra.run.smithi023.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2023-03-27T09:59:27.572 INFO:teuthology.orchestra.run.smithi023.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 11 MB/s | 1.5 MB 00:00 2023-03-27T09:59:27.597 INFO:teuthology.orchestra.run.smithi023.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 2.3 MB/s | 200 kB 00:00 2023-03-27T09:59:27.622 INFO:teuthology.orchestra.run.smithi023.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 849 kB/s | 43 kB 00:00 2023-03-27T09:59:27.656 INFO:teuthology.orchestra.run.smithi023.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 3.7 MB/s | 220 kB 00:00 2023-03-27T09:59:27.681 INFO:teuthology.orchestra.run.smithi023.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 821 kB/s | 48 kB 00:00 2023-03-27T09:59:27.765 INFO:teuthology.orchestra.run.smithi023.stdout:(107/131): postgresql-server-10.21-2.module+el8 8.6 MB/s | 5.1 MB 00:00 2023-03-27T09:59:27.782 INFO:teuthology.orchestra.run.smithi023.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 425 kB/s | 43 kB 00:00 2023-03-27T09:59:27.807 INFO:teuthology.orchestra.run.smithi023.stdout:(109/131): libsepol-devel-2.9-3.el8.x86_64.rpm 577 kB/s | 87 kB 00:00 2023-03-27T09:59:27.832 INFO:teuthology.orchestra.run.smithi023.stdout:(110/131): python3-pycparser-2.14-14.el8.noarch 1.6 MB/s | 109 kB 00:00 2023-03-27T09:59:27.857 INFO:teuthology.orchestra.run.smithi023.stdout:(111/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.0 MB/s | 229 kB 00:00 2023-03-27T09:59:27.874 INFO:teuthology.orchestra.run.smithi023.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 445 kB/s | 19 kB 00:00 2023-03-27T09:59:27.899 INFO:teuthology.orchestra.run.smithi023.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.6 MB/s | 155 kB 00:00 2023-03-27T09:59:27.942 INFO:teuthology.orchestra.run.smithi023.stdout:(114/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 4.1 MB/s | 560 kB 00:00 2023-03-27T09:59:27.967 INFO:teuthology.orchestra.run.smithi023.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 267 kB/s | 18 kB 00:00 2023-03-27T09:59:28.009 INFO:teuthology.orchestra.run.smithi023.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 4.1 MB/s | 559 kB 00:00 2023-03-27T09:59:28.043 INFO:teuthology.orchestra.run.smithi023.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.1 MB/s | 238 kB 00:00 2023-03-27T09:59:28.076 INFO:teuthology.orchestra.run.smithi023.stdout:(118/131): pcre2-devel-10.32-2.el8.x86_64.rpm 4.4 MB/s | 605 kB 00:00 2023-03-27T09:59:28.102 INFO:teuthology.orchestra.run.smithi023.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.2 MB/s | 72 kB 00:00 2023-03-27T09:59:28.127 INFO:teuthology.orchestra.run.smithi023.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-03-27T09:59:28.152 INFO:teuthology.orchestra.run.smithi023.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-03-27T09:59:28.186 INFO:teuthology.orchestra.run.smithi023.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.6 MB/s | 98 kB 00:00 2023-03-27T09:59:28.211 INFO:teuthology.orchestra.run.smithi023.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 984 kB/s | 58 kB 00:00 2023-03-27T09:59:28.236 INFO:teuthology.orchestra.run.smithi023.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 770 kB/s | 39 kB 00:00 2023-03-27T09:59:28.278 INFO:teuthology.orchestra.run.smithi023.stdout:(125/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 4.0 MB/s | 171 kB 00:00 2023-03-27T09:59:28.329 INFO:teuthology.orchestra.run.smithi023.stdout:(126/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 5.4 MB/s | 644 kB 00:00 2023-03-27T09:59:28.354 INFO:teuthology.orchestra.run.smithi023.stdout:(127/131): NetworkManager-initscripts-updown-1. 1.8 MB/s | 138 kB 00:00 2023-03-27T09:59:28.429 INFO:teuthology.orchestra.run.smithi023.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 5.5 MB/s | 2.3 MB 00:00 2023-03-27T09:59:28.488 INFO:teuthology.orchestra.run.smithi023.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-03-27T09:59:28.730 INFO:teuthology.orchestra.run.smithi023.stdout:(130/131): librados2-18.0.0-2533.gb70fdf97.el8. 9.1 MB/s | 3.6 MB 00:00 2023-03-27T09:59:28.905 INFO:teuthology.orchestra.run.smithi023.stdout:(131/131): librbd1-18.0.0-2533.gb70fdf97.el8.x8 6.5 MB/s | 3.6 MB 00:00 2023-03-27T09:59:28.908 INFO:teuthology.orchestra.run.smithi023.stdout:-------------------------------------------------------------------------------- 2023-03-27T09:59:28.908 INFO:teuthology.orchestra.run.smithi023.stdout:Total 13 MB/s | 199 MB 00:15 2023-03-27T09:59:30.420 INFO:teuthology.orchestra.run.smithi023.stdout:Running transaction check 2023-03-27T09:59:30.566 INFO:teuthology.orchestra.run.smithi023.stdout:Transaction check succeeded. 2023-03-27T09:59:30.566 INFO:teuthology.orchestra.run.smithi023.stdout:Running transaction test 2023-03-27T09:59:31.799 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-03-27T09:59:31.980 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-03-27T09:59:32.167 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-03-27T09:59:32.339 INFO:teuthology.orchestra.run.smithi023.stdout:Transaction test succeeded. 2023-03-27T09:59:32.344 INFO:teuthology.orchestra.run.smithi023.stdout:Running transaction 2023-03-27T09:59:32.380 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-03-27T09:59:32.574 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-03-27T09:59:32.695 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-03-27T09:59:32.932 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-03-27T09:59:33.322 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-03-27T09:59:33.548 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librados2-2:18.0.0-2533.gb70fdf97.el8.x86_64 6/134 2023-03-27T09:59:33.942 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:18.0.0-2533.gb70fdf97.el8.x86_64 6/134 2023-03-27T09:59:34.115 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librbd1-2:18.0.0-2533.gb70fdf97.el8.x86_64 7/134 2023-03-27T09:59:34.243 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:18.0.0-2533.gb70fdf97.el8.x86_64 7/134 2023-03-27T09:59:34.458 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs2-2:18.0.0-2533.gb70fdf97.el8.x86_64 8/134 2023-03-27T09:59:34.546 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephfs2-2:18.0.0-2533.gb70fdf97.el8.x86_64 8/134 2023-03-27T09:59:34.765 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rados-2:18.0.0-2533.gb70fdf97.el8.x86_64 9/134 2023-03-27T09:59:34.976 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-03-27T09:59:35.226 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-03-27T09:59:35.667 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-03-27T09:59:35.935 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-03-27T09:59:36.131 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephsqlite-2:18.0.0-2533.gb70fdf97.el8.x86_64 14/134 2023-03-27T09:59:36.253 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephsqlite-2:18.0.0-2533.gb70fdf97.el8.x86_64 14/134 2023-03-27T09:59:36.440 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libradosstriper1-2:18.0.0-2533.gb70fdf97.el8.x86 15/134 2023-03-27T09:59:36.523 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libradosstriper1-2:18.0.0-2533.gb70fdf97.el8.x86 15/134 2023-03-27T09:59:36.689 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-03-27T09:59:36.819 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-03-27T09:59:36.981 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-03-27T09:59:37.293 INFO:teuthology.orchestra.run.smithi023.stdout: Preparing : 1/1 2023-03-27T09:59:37.498 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-03-27T09:59:37.593 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-03-27T09:59:37.681 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-03-27T09:59:37.781 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-03-27T09:59:37.878 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-03-27T09:59:38.057 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-03-27T09:59:38.207 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-03-27T09:59:38.368 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-03-27T09:59:38.424 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-03-27T09:59:38.781 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-03-27T09:59:38.933 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-03-27T09:59:39.148 INFO:teuthology.orchestra.run.smithi023.stdout: Upgrading : librados2-2:18.0.0-2533.gb70fdf97.el8.x86_64 6/134 2023-03-27T09:59:39.509 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-03-27T09:59:39.614 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: librados2-2:18.0.0-2533.gb70fdf97.el8.x86_64 6/134 2023-03-27T09:59:39.774 INFO:teuthology.orchestra.run.smithi023.stdout: Upgrading : librbd1-2:18.0.0-2533.gb70fdf97.el8.x86_64 7/134 2023-03-27T09:59:39.796 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-03-27T09:59:39.924 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: librbd1-2:18.0.0-2533.gb70fdf97.el8.x86_64 7/134 2023-03-27T09:59:39.999 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-03-27T09:59:40.107 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : libcephfs2-2:18.0.0-2533.gb70fdf97.el8.x86_64 8/134 2023-03-27T09:59:40.243 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-03-27T09:59:40.385 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: libcephfs2-2:18.0.0-2533.gb70fdf97.el8.x86_64 8/134 2023-03-27T09:59:40.453 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-03-27T09:59:40.597 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-rados-2:18.0.0-2533.gb70fdf97.el8.x86_64 9/134 2023-03-27T09:59:40.663 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-03-27T09:59:40.869 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-03-27T09:59:40.899 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-03-27T09:59:41.070 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-common-2:18.0.0-2533.gb70fdf97.el8. 26/134 2023-03-27T09:59:41.071 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-03-27T09:59:41.216 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-argparse-2:18.0.0-2533.gb70fdf97.el 27/134 2023-03-27T09:59:41.541 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-03-27T09:59:41.792 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-03-27T09:59:42.115 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : libcephsqlite-2:18.0.0-2533.gb70fdf97.el8.x86_64 14/134 2023-03-27T09:59:42.129 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cephfs-2:18.0.0-2533.gb70fdf97.el8.x86_6 28/134 2023-03-27T09:59:42.250 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: libcephsqlite-2:18.0.0-2533.gb70fdf97.el8.x86_64 14/134 2023-03-27T09:59:42.423 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : libradosstriper1-2:18.0.0-2533.gb70fdf97.el8.x86 15/134 2023-03-27T09:59:42.478 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-03-27T09:59:42.511 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: libradosstriper1-2:18.0.0-2533.gb70fdf97.el8.x86 15/134 2023-03-27T09:59:42.656 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-03-27T09:59:42.775 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-03-27T09:59:42.820 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-03-27T09:59:42.956 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-03-27T09:59:43.020 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-03-27T09:59:43.213 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-03-27T09:59:43.703 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-03-27T09:59:43.865 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-03-27T09:59:44.592 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-03-27T09:59:44.781 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-03-27T09:59:44.974 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-03-27T09:59:45.316 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-03-27T09:59:45.517 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-03-27T09:59:45.674 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-03-27T09:59:45.733 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-03-27T09:59:45.911 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rbd-2:18.0.0-2533.gb70fdf97.el8.x86_64 36/134 2023-03-27T09:59:45.988 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-03-27T09:59:46.084 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librados-devel-2:18.0.0-2533.gb70fdf97.el8.x86_6 37/134 2023-03-27T09:59:46.224 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-03-27T09:59:46.258 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-03-27T09:59:46.405 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-03-27T09:59:46.481 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-03-27T09:59:46.582 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-03-27T09:59:46.650 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-03-27T09:59:46.724 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-03-27T09:59:46.871 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-03-27T09:59:46.942 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-03-27T09:59:47.120 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-03-27T09:59:47.165 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-03-27T09:59:47.280 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-ceph-common-2:18.0.0-2533.gb70fdf97.el8. 26/134 2023-03-27T09:59:47.428 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-ceph-argparse-2:18.0.0-2533.gb70fdf97.el 27/134 2023-03-27T09:59:47.512 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-03-27T09:59:47.791 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-03-27T09:59:48.031 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-03-27T09:59:48.240 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-03-27T09:59:48.467 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-03-27T09:59:48.526 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-cephfs-2:18.0.0-2533.gb70fdf97.el8.x86_6 28/134 2023-03-27T09:59:48.804 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-03-27T09:59:48.895 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-03-27T09:59:49.079 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-03-27T09:59:49.167 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-03-27T09:59:49.443 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-03-27T09:59:49.647 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-03-27T09:59:49.649 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-03-27T09:59:49.928 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-03-27T09:59:50.088 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-03-27T09:59:50.283 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-03-27T09:59:50.587 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-03-27T09:59:50.852 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-03-27T09:59:51.126 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-03-27T09:59:51.377 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-03-27T09:59:51.545 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-03-27T09:59:51.730 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-03-27T09:59:51.838 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-03-27T09:59:51.929 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-03-27T09:59:52.003 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-03-27T09:59:52.225 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-03-27T09:59:52.260 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-03-27T09:59:52.445 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-03-27T09:59:52.491 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-03-27T09:59:52.638 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-03-27T09:59:52.654 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-03-27T09:59:52.731 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-03-27T09:59:52.831 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-rbd-2:18.0.0-2533.gb70fdf97.el8.x86_64 36/134 2023-03-27T09:59:52.966 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-03-27T09:59:53.010 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : librados-devel-2:18.0.0-2533.gb70fdf97.el8.x86_6 37/134 2023-03-27T09:59:53.138 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-03-27T09:59:53.198 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-03-27T09:59:53.328 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-03-27T09:59:53.409 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-03-27T09:59:53.506 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-03-27T09:59:53.700 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-03-27T09:59:53.809 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-03-27T09:59:53.855 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-03-27T09:59:54.052 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 70/134 2023-03-27T09:59:54.065 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-03-27T09:59:54.191 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-03-27T09:59:54.283 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-03-27T09:59:54.364 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-03-27T09:59:54.545 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-03-27T09:59:54.687 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-03-27T09:59:54.790 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-03-27T09:59:54.964 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-03-27T09:59:54.997 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-03-27T09:59:55.157 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-03-27T09:59:55.203 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-03-27T09:59:55.293 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-03-27T09:59:55.463 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-03-27T09:59:55.525 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-03-27T09:59:55.708 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-03-27T09:59:55.776 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-03-27T09:59:55.918 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-03-27T09:59:56.107 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-03-27T09:59:56.169 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-03-27T09:59:56.349 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-03-27T09:59:56.411 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-03-27T09:59:56.517 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-03-27T09:59:56.617 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-03-27T09:59:56.743 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-03-27T09:59:56.822 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-03-27T09:59:56.948 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-03-27T09:59:57.134 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-03-27T09:59:57.491 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-03-27T09:59:57.801 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-03-27T09:59:57.935 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-03-27T09:59:58.083 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-03-27T09:59:58.190 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-03-27T09:59:58.330 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-03-27T09:59:58.428 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-03-27T09:59:58.506 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-03-27T09:59:58.699 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-03-27T09:59:58.737 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-03-27T09:59:58.896 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-03-27T09:59:58.977 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-03-27T09:59:59.036 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-03-27T09:59:59.223 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-03-27T09:59:59.257 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-03-27T09:59:59.453 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-03-27T09:59:59.613 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-03-27T09:59:59.714 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-03-27T09:59:59.962 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-03-27T10:00:00.047 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-03-27T10:00:00.141 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-03-27T10:00:00.203 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librgw2-2:18.0.0-2533.gb70fdf97.el8.x86_64 92/134 2023-03-27T10:00:00.285 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librgw2-2:18.0.0-2533.gb70fdf97.el8.x86_64 92/134 2023-03-27T10:00:00.314 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-03-27T10:00:00.487 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rgw-2:18.0.0-2533.gb70fdf97.el8.x86_64 93/134 2023-03-27T10:00:00.513 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-03-27T10:00:00.855 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-03-27T10:00:01.104 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 70/134 2023-03-27T10:00:01.268 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-03-27T10:00:01.426 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-03-27T10:00:01.622 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-03-27T10:00:01.822 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-03-27T10:00:02.037 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-03-27T10:00:02.191 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-03-27T10:00:02.336 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-03-27T10:00:02.556 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-03-27T10:00:02.768 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-03-27T10:00:02.961 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-03-27T10:00:03.251 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-03-27T10:00:03.497 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-03-27T10:00:03.673 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-03-27T10:00:03.888 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-03-27T10:00:03.990 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:18.0.0-2533.gb70fdf97.el8.x86_64 94/134 2023-03-27T10:00:04.211 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-common-2:18.0.0-2533.gb70fdf97.el8.x86_64 94/134 2023-03-27T10:00:04.765 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:18.0.0-2533.gb70fdf97.el8.x86_64 94/134 2023-03-27T10:00:05.253 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-03-27T10:00:05.526 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-03-27T10:00:05.572 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-base-2:18.0.0-2533.gb70fdf97.el8.x86_64 95/134 2023-03-27T10:00:05.691 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-base-2:18.0.0-2533.gb70fdf97.el8.x86_64 95/134 2023-03-27T10:00:05.778 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-03-27T10:00:05.836 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-selinux-2:18.0.0-2533.gb70fdf97.el8.x86_64 96/134 2023-03-27T10:00:06.104 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-03-27T10:00:06.369 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-03-27T10:00:06.584 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-03-27T10:00:07.472 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-03-27T10:00:07.661 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : librgw2-2:18.0.0-2533.gb70fdf97.el8.x86_64 92/134 2023-03-27T10:00:07.756 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: librgw2-2:18.0.0-2533.gb70fdf97.el8.x86_64 92/134 2023-03-27T10:00:07.929 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-rgw-2:18.0.0-2533.gb70fdf97.el8.x86_64 93/134 2023-03-27T10:00:12.766 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: ceph-common-2:18.0.0-2533.gb70fdf97.el8.x86_64 94/134 2023-03-27T10:00:13.028 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : ceph-common-2:18.0.0-2533.gb70fdf97.el8.x86_64 94/134 2023-03-27T10:00:13.702 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: ceph-common-2:18.0.0-2533.gb70fdf97.el8.x86_64 94/134 2023-03-27T10:00:14.009 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : ceph-base-2:18.0.0-2533.gb70fdf97.el8.x86_64 95/134 2023-03-27T10:00:14.131 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: ceph-base-2:18.0.0-2533.gb70fdf97.el8.x86_64 95/134 2023-03-27T10:00:14.294 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : ceph-selinux-2:18.0.0-2533.gb70fdf97.el8.x86_64 96/134 2023-03-27T10:00:24.969 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-selinux-2:18.0.0-2533.gb70fdf97.el8.x86_64 96/134 2023-03-27T10:00:24.969 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /sys 2023-03-27T10:00:24.970 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /proc 2023-03-27T10:00:24.970 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /dev 2023-03-27T10:00:24.970 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /run 2023-03-27T10:00:24.970 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /mnt 2023-03-27T10:00:24.970 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /var/tmp 2023-03-27T10:00:24.970 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /home 2023-03-27T10:00:24.970 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /tmp 2023-03-27T10:00:24.971 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /dev 2023-03-27T10:00:24.971 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-03-27T10:00:25.325 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-volume-2:18.0.0-2533.gb70fdf97.el8.noarch 97/134 2023-03-27T10:00:26.774 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-volume-2:18.0.0-2533.gb70fdf97.el8.noarch 97/134 2023-03-27T10:00:26.952 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-osd-2:18.0.0-2533.gb70fdf97.el8.x86_64 98/134 2023-03-27T10:00:27.169 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-osd-2:18.0.0-2533.gb70fdf97.el8.x86_64 98/134 2023-03-27T10:00:27.325 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mds-2:18.0.0-2533.gb70fdf97.el8.x86_64 99/134 2023-03-27T10:00:27.704 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mds-2:18.0.0-2533.gb70fdf97.el8.x86_64 99/134 2023-03-27T10:00:27.900 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mon-2:18.0.0-2533.gb70fdf97.el8.x86_64 100/134 2023-03-27T10:00:27.917 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mon-2:18.0.0-2533.gb70fdf97.el8.x86_64 100/134 2023-03-27T10:00:29.481 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephadm-2:18.0.0-2533.gb70fdf97.el8.noarch 101/134 2023-03-27T10:00:29.713 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephadm-2:18.0.0-2533.gb70fdf97.el8.noarch 101/134 2023-03-27T10:00:29.839 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-2533.gb70fdf97.e 102/134 2023-03-27T10:00:30.078 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-2533.gb70fdf97. 103/134 2023-03-27T10:00:30.266 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-2533.gb70fdf97.el8.noa 104/134 2023-03-27T10:00:30.483 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-2533.gb70fdf97.el8.noa 104/134 2023-03-27T10:00:30.735 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-2533.gb70fdf97.el8.n 105/134 2023-03-27T10:00:31.690 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-2533.gb70fdf97.el8.n 105/134 2023-03-27T10:00:31.840 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-2533.gb70 106/134 2023-03-27T10:00:31.929 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-2533.gb70 106/134 2023-03-27T10:00:32.279 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-2533.gb70fdf97.el 107/134 2023-03-27T10:00:32.460 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-2:18.0.0-2533.gb70fdf97.el8.x86_64 108/134 2023-03-27T10:00:32.495 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-2:18.0.0-2533.gb70fdf97.el8.x86_64 108/134 2023-03-27T10:00:32.682 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-rook-2:18.0.0-2533.gb70fdf97.el8.noarch 109/134 2023-03-27T10:00:32.693 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-2533.gb70fdf97.el8.noarch 109/134 2023-03-27T10:00:33.952 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-2:18.0.0-2533.gb70fdf97.el8.x86_64 110/134 2023-03-27T10:00:34.192 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-radosgw-2:18.0.0-2533.gb70fdf97.el8.x86_64 111/134 2023-03-27T10:00:34.225 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-2533.gb70fdf97.el8.x86_64 111/134 2023-03-27T10:00:34.382 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-2533.gb70fd 112/134 2023-03-27T10:00:34.423 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-2533.gb70fd 112/134 2023-03-27T10:00:34.582 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-mirror-2:18.0.0-2533.gb70fdf97.el8.x86_64 113/134 2023-03-27T10:00:34.926 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-2533.gb70fdf97.el8.x86_64 113/134 2023-03-27T10:00:35.090 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-mirror-2:18.0.0-2533.gb70fdf97.el8.x86_64 114/134 2023-03-27T10:00:35.212 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: ceph-selinux-2:18.0.0-2533.gb70fdf97.el8.x86_64 96/134 2023-03-27T10:00:35.213 INFO:teuthology.orchestra.run.smithi023.stdout:skipping the directory /sys 2023-03-27T10:00:35.213 INFO:teuthology.orchestra.run.smithi023.stdout:skipping the directory /proc 2023-03-27T10:00:35.213 INFO:teuthology.orchestra.run.smithi023.stdout:skipping the directory /dev 2023-03-27T10:00:35.213 INFO:teuthology.orchestra.run.smithi023.stdout:skipping the directory /run 2023-03-27T10:00:35.214 INFO:teuthology.orchestra.run.smithi023.stdout:skipping the directory /mnt 2023-03-27T10:00:35.214 INFO:teuthology.orchestra.run.smithi023.stdout:skipping the directory /var/tmp 2023-03-27T10:00:35.214 INFO:teuthology.orchestra.run.smithi023.stdout:skipping the directory /home 2023-03-27T10:00:35.214 INFO:teuthology.orchestra.run.smithi023.stdout:skipping the directory /tmp 2023-03-27T10:00:35.214 INFO:teuthology.orchestra.run.smithi023.stdout:skipping the directory /dev 2023-03-27T10:00:35.214 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:00:35.599 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : ceph-volume-2:18.0.0-2533.gb70fdf97.el8.noarch 97/134 2023-03-27T10:00:37.302 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: ceph-volume-2:18.0.0-2533.gb70fdf97.el8.noarch 97/134 2023-03-27T10:00:37.480 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : ceph-osd-2:18.0.0-2533.gb70fdf97.el8.x86_64 98/134 2023-03-27T10:00:37.758 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: ceph-osd-2:18.0.0-2533.gb70fdf97.el8.x86_64 98/134 2023-03-27T10:00:37.916 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : ceph-mds-2:18.0.0-2533.gb70fdf97.el8.x86_64 99/134 2023-03-27T10:00:38.384 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: ceph-mds-2:18.0.0-2533.gb70fdf97.el8.x86_64 99/134 2023-03-27T10:00:38.582 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : ceph-mon-2:18.0.0-2533.gb70fdf97.el8.x86_64 100/134 2023-03-27T10:00:38.620 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: ceph-mon-2:18.0.0-2533.gb70fdf97.el8.x86_64 100/134 2023-03-27T10:00:39.357 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: rbd-mirror-2:18.0.0-2533.gb70fdf97.el8.x86_64 114/134 2023-03-27T10:00:39.620 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-test-2:18.0.0-2533.gb70fdf97.el8.x86_64 115/134 2023-03-27T10:00:39.806 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-03-27T10:00:39.998 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-03-27T10:00:40.890 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-03-27T10:00:41.204 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-03-27T10:00:41.206 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: cephadm-2:18.0.0-2533.gb70fdf97.el8.noarch 101/134 2023-03-27T10:00:41.374 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : cephadm-2:18.0.0-2533.gb70fdf97.el8.noarch 101/134 2023-03-27T10:00:41.380 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-fuse-2:18.0.0-2533.gb70fdf97.el8.x86_64 119/134 2023-03-27T10:00:41.540 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-2533.gb70fdf97.e 102/134 2023-03-27T10:00:41.593 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-03-27T10:00:41.751 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-2533.gb70fdf97. 103/134 2023-03-27T10:00:41.768 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs-devel-2:18.0.0-2533.gb70fdf97.el8.x86_ 121/134 2023-03-27T10:00:41.940 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-03-27T10:00:41.948 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-2533.gb70fdf97.el8.noa 104/134 2023-03-27T10:00:42.216 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-2533.gb70fdf97.el8.noa 104/134 2023-03-27T10:00:42.404 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-2533.gb70fdf97.el8.n 105/134 2023-03-27T10:00:43.588 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-2533.gb70fdf97.el8.n 105/134 2023-03-27T10:00:43.748 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-2533.gb70 106/134 2023-03-27T10:00:43.877 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-2533.gb70 106/134 2023-03-27T10:00:44.077 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-03-27T10:00:44.263 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-2533.gb70fdf97.el 107/134 2023-03-27T10:00:44.348 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-03-27T10:00:44.369 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-03-27T10:00:44.455 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : ceph-mgr-2:18.0.0-2533.gb70fdf97.el8.x86_64 108/134 2023-03-27T10:00:44.517 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: ceph-mgr-2:18.0.0-2533.gb70fdf97.el8.x86_64 108/134 2023-03-27T10:00:44.559 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-top-2:18.0.0-2533.gb70fdf97.el8.noarch 124/134 2023-03-27T10:00:44.715 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : ceph-mgr-rook-2:18.0.0-2533.gb70fdf97.el8.noarch 109/134 2023-03-27T10:00:44.740 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-2533.gb70fdf97.el8.noarch 109/134 2023-03-27T10:00:44.797 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-fuse-2:18.0.0-2533.gb70fdf97.el8.x86_64 125/134 2023-03-27T10:00:45.113 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-nbd-2:18.0.0-2533.gb70fdf97.el8.x86_64 126/134 2023-03-27T10:00:45.441 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-03-27T10:00:45.581 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-03-27T10:00:45.589 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-03-27T10:00:45.780 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-03-27T10:00:45.938 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-03-27T10:00:46.025 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-03-27T10:00:46.216 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : ceph-2:18.0.0-2533.gb70fdf97.el8.x86_64 110/134 2023-03-27T10:00:46.222 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-03-27T10:00:46.234 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-03-27T10:00:46.234 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-03-27T10:00:46.450 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-03-27T10:00:46.451 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-03-27T10:00:46.475 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : ceph-radosgw-2:18.0.0-2533.gb70fdf97.el8.x86_64 111/134 2023-03-27T10:00:46.533 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-2533.gb70fdf97.el8.x86_64 111/134 2023-03-27T10:00:46.673 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-03-27T10:00:46.706 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-2533.gb70fd 112/134 2023-03-27T10:00:46.774 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-2533.gb70fd 112/134 2023-03-27T10:00:46.871 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-03-27T10:00:46.939 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : cephfs-mirror-2:18.0.0-2533.gb70fdf97.el8.x86_64 113/134 2023-03-27T10:00:47.152 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-03-27T10:00:47.152 INFO:teuthology.orchestra.run.smithi112.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-03-27T10:00:47.152 INFO:teuthology.orchestra.run.smithi112.stdout:All rules containing unresolvable specifiers will be skipped. 2023-03-27T10:00:47.152 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-03-27T10:00:47.368 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-2533.gb70fdf97.el8.x86_64 113/134 2023-03-27T10:00:47.374 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-2:18.0.0-2533.gb70fdf97.el8.x86_64 1/134 2023-03-27T10:00:47.375 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-base-2:18.0.0-2533.gb70fdf97.el8.x86_64 2/134 2023-03-27T10:00:47.375 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-common-2:18.0.0-2533.gb70fdf97.el8.x86_64 3/134 2023-03-27T10:00:47.375 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-fuse-2:18.0.0-2533.gb70fdf97.el8.x86_64 4/134 2023-03-27T10:00:47.375 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-2533.gb70fd 5/134 2023-03-27T10:00:47.375 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mds-2:18.0.0-2533.gb70fdf97.el8.x86_64 6/134 2023-03-27T10:00:47.375 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-2:18.0.0-2533.gb70fdf97.el8.x86_64 7/134 2023-03-27T10:00:47.376 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mon-2:18.0.0-2533.gb70fdf97.el8.x86_64 8/134 2023-03-27T10:00:47.376 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-osd-2:18.0.0-2533.gb70fdf97.el8.x86_64 9/134 2023-03-27T10:00:47.376 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-radosgw-2:18.0.0-2533.gb70fdf97.el8.x86_64 10/134 2023-03-27T10:00:47.376 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-selinux-2:18.0.0-2533.gb70fdf97.el8.x86_64 11/134 2023-03-27T10:00:47.376 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-test-2:18.0.0-2533.gb70fdf97.el8.x86_64 12/134 2023-03-27T10:00:47.376 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-mirror-2:18.0.0-2533.gb70fdf97.el8.x86_64 13/134 2023-03-27T10:00:47.376 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs-devel-2:18.0.0-2533.gb70fdf97.el8.x86_ 14/134 2023-03-27T10:00:47.376 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs2-2:18.0.0-2533.gb70fdf97.el8.x86_64 15/134 2023-03-27T10:00:47.377 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephsqlite-2:18.0.0-2533.gb70fdf97.el8.x86_64 16/134 2023-03-27T10:00:47.377 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados-devel-2:18.0.0-2533.gb70fdf97.el8.x86_6 17/134 2023-03-27T10:00:47.377 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libradosstriper1-2:18.0.0-2533.gb70fdf97.el8.x86 18/134 2023-03-27T10:00:47.377 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librgw2-2:18.0.0-2533.gb70fdf97.el8.x86_64 19/134 2023-03-27T10:00:47.377 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-argparse-2:18.0.0-2533.gb70fdf97.el 20/134 2023-03-27T10:00:47.377 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-common-2:18.0.0-2533.gb70fdf97.el8. 21/134 2023-03-27T10:00:47.377 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cephfs-2:18.0.0-2533.gb70fdf97.el8.x86_6 22/134 2023-03-27T10:00:47.378 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rados-2:18.0.0-2533.gb70fdf97.el8.x86_64 23/134 2023-03-27T10:00:47.378 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rbd-2:18.0.0-2533.gb70fdf97.el8.x86_64 24/134 2023-03-27T10:00:47.378 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rgw-2:18.0.0-2533.gb70fdf97.el8.x86_64 25/134 2023-03-27T10:00:47.385 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-fuse-2:18.0.0-2533.gb70fdf97.el8.x86_64 26/134 2023-03-27T10:00:47.385 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-mirror-2:18.0.0-2533.gb70fdf97.el8.x86_64 27/134 2023-03-27T10:00:47.385 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-nbd-2:18.0.0-2533.gb70fdf97.el8.x86_64 28/134 2023-03-27T10:00:47.385 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-2533.gb70fdf97. 29/134 2023-03-27T10:00:47.386 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-2533.gb70fdf97.el8.noa 30/134 2023-03-27T10:00:47.386 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-2533.gb70fdf97.el8.n 31/134 2023-03-27T10:00:47.386 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-2533.gb70 32/134 2023-03-27T10:00:47.386 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-2533.gb70fdf97.el 33/134 2023-03-27T10:00:47.386 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-rook-2:18.0.0-2533.gb70fdf97.el8.noarch 34/134 2023-03-27T10:00:47.386 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-2533.gb70fdf97.e 35/134 2023-03-27T10:00:47.386 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-volume-2:18.0.0-2533.gb70fdf97.el8.noarch 36/134 2023-03-27T10:00:47.387 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephadm-2:18.0.0-2533.gb70fdf97.el8.noarch 37/134 2023-03-27T10:00:47.387 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-top-2:18.0.0-2533.gb70fdf97.el8.noarch 38/134 2023-03-27T10:00:47.387 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-03-27T10:00:47.387 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-03-27T10:00:47.387 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-03-27T10:00:47.387 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-03-27T10:00:47.387 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-03-27T10:00:47.387 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-03-27T10:00:47.388 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-03-27T10:00:47.388 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-03-27T10:00:47.388 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-03-27T10:00:47.388 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-03-27T10:00:47.388 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-03-27T10:00:47.388 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-03-27T10:00:47.388 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-03-27T10:00:47.389 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-03-27T10:00:47.389 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-03-27T10:00:47.389 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-03-27T10:00:47.389 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-03-27T10:00:47.389 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-03-27T10:00:47.389 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-03-27T10:00:47.389 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-03-27T10:00:47.389 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-03-27T10:00:47.390 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-03-27T10:00:47.390 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-03-27T10:00:47.390 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-03-27T10:00:47.390 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-03-27T10:00:47.390 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-03-27T10:00:47.390 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 65/134 2023-03-27T10:00:47.390 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-03-27T10:00:47.391 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-03-27T10:00:47.391 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-03-27T10:00:47.391 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-03-27T10:00:47.391 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-03-27T10:00:47.391 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-03-27T10:00:47.391 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-03-27T10:00:47.391 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-03-27T10:00:47.391 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-03-27T10:00:47.392 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-03-27T10:00:47.392 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-03-27T10:00:47.392 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-03-27T10:00:47.392 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-03-27T10:00:47.392 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-03-27T10:00:47.392 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-03-27T10:00:47.392 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-03-27T10:00:47.392 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-03-27T10:00:47.393 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-03-27T10:00:47.393 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-03-27T10:00:47.393 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-03-27T10:00:47.393 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-03-27T10:00:47.393 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-03-27T10:00:47.393 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-03-27T10:00:47.393 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-03-27T10:00:47.393 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-03-27T10:00:47.394 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-03-27T10:00:47.394 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-03-27T10:00:47.394 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-03-27T10:00:47.394 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-03-27T10:00:47.394 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-03-27T10:00:47.394 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-03-27T10:00:47.394 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-03-27T10:00:47.395 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-03-27T10:00:47.395 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-03-27T10:00:47.395 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-03-27T10:00:47.395 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-03-27T10:00:47.395 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-03-27T10:00:47.395 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-03-27T10:00:47.395 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-03-27T10:00:47.395 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-03-27T10:00:47.396 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-03-27T10:00:47.396 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-03-27T10:00:47.396 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-03-27T10:00:47.396 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-03-27T10:00:47.396 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-03-27T10:00:47.396 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-03-27T10:00:47.396 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-03-27T10:00:47.396 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-03-27T10:00:47.397 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-03-27T10:00:47.397 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-03-27T10:00:47.397 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-03-27T10:00:47.397 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-03-27T10:00:47.397 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-03-27T10:00:47.397 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-03-27T10:00:47.397 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-03-27T10:00:47.398 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-03-27T10:00:47.398 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-03-27T10:00:47.398 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-03-27T10:00:47.398 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-03-27T10:00:47.398 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-03-27T10:00:47.398 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-03-27T10:00:47.398 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-03-27T10:00:47.398 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-03-27T10:00:47.399 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:18.0.0-2533.gb70fdf97.el8.x86_64 129/134 2023-03-27T10:00:47.399 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-03-27T10:00:47.399 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:18.0.0-2533.gb70fdf97.el8.x86_64 131/134 2023-03-27T10:00:47.399 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-03-27T10:00:47.399 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-03-27T10:00:47.556 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : rbd-mirror-2:18.0.0-2533.gb70fdf97.el8.x86_64 114/134 2023-03-27T10:00:50.047 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-03-27T10:00:50.047 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:01:51 ago on Mon 27 Mar 2023 09:58:59 AM UTC. 2023-03-27T10:00:52.587 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: rbd-mirror-2:18.0.0-2533.gb70fdf97.el8.x86_64 114/134 2023-03-27T10:00:52.730 INFO:teuthology.orchestra.run.smithi112.stdout:Installed products updated. 2023-03-27T10:00:52.937 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : ceph-test-2:18.0.0-2533.gb70fdf97.el8.x86_64 115/134 2023-03-27T10:00:52.963 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-03-27T10:00:52.963 INFO:teuthology.orchestra.run.smithi112.stdout:Upgraded: 2023-03-27T10:00:52.963 INFO:teuthology.orchestra.run.smithi112.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-03-27T10:00:52.963 INFO:teuthology.orchestra.run.smithi112.stdout: librados2-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:00:52.963 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:00:52.964 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-03-27T10:00:52.964 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-03-27T10:00:52.964 INFO:teuthology.orchestra.run.smithi112.stdout: automake-1.16.1-7.el8.noarch 2023-03-27T10:00:52.964 INFO:teuthology.orchestra.run.smithi112.stdout: bison-3.0.4-10.el8.x86_64 2023-03-27T10:00:52.964 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:00:52.964 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:00:52.964 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:00:52.965 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:00:52.965 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards-2:18.0.0-2533.gb70fdf97.el8.noarch 2023-03-27T10:00:52.965 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:00:52.965 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:00:52.965 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:00:52.965 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm-2:18.0.0-2533.gb70fdf97.el8.noarch 2023-03-27T10:00:52.965 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard-2:18.0.0-2533.gb70fdf97.el8.noarch 2023-03-27T10:00:52.966 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local-2:18.0.0-2533.gb70fdf97.el8.noarch 2023-03-27T10:00:52.966 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core-2:18.0.0-2533.gb70fdf97.el8.noarch 2023-03-27T10:00:52.966 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook-2:18.0.0-2533.gb70fdf97.el8.noarch 2023-03-27T10:00:52.966 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:00:52.966 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:00:52.966 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts-2:18.0.0-2533.gb70fdf97.el8.noarch 2023-03-27T10:00:52.966 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:00:52.966 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:00:52.967 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:00:52.967 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume-2:18.0.0-2533.gb70fdf97.el8.noarch 2023-03-27T10:00:52.967 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm-2:18.0.0-2533.gb70fdf97.el8.noarch 2023-03-27T10:00:52.967 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:00:52.967 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-top-2:18.0.0-2533.gb70fdf97.el8.noarch 2023-03-27T10:00:52.967 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-03-27T10:00:52.968 INFO:teuthology.orchestra.run.smithi112.stdout: flex-2.6.1-9.el8.x86_64 2023-03-27T10:00:52.968 INFO:teuthology.orchestra.run.smithi112.stdout: fuse-2.9.7-15.el8.x86_64 2023-03-27T10:00:52.968 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-03-27T10:00:52.968 INFO:teuthology.orchestra.run.smithi112.stdout: jq-1.6-3.el8.x86_64 2023-03-27T10:00:52.968 INFO:teuthology.orchestra.run.smithi112.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-03-27T10:00:52.968 INFO:teuthology.orchestra.run.smithi112.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-03-27T10:00:52.968 INFO:teuthology.orchestra.run.smithi112.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-03-27T10:00:52.968 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:00:52.969 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:00:52.969 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:00:52.969 INFO:teuthology.orchestra.run.smithi112.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-03-27T10:00:52.969 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-03-27T10:00:52.969 INFO:teuthology.orchestra.run.smithi112.stdout: liboath-2.6.2-3.el8.x86_64 2023-03-27T10:00:52.969 INFO:teuthology.orchestra.run.smithi112.stdout: libpq-13.5-1.el8.x86_64 2023-03-27T10:00:52.969 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-03-27T10:00:52.970 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-03-27T10:00:52.970 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:00:52.970 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:00:52.970 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-03-27T10:00:52.970 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:00:52.970 INFO:teuthology.orchestra.run.smithi112.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-03-27T10:00:52.970 INFO:teuthology.orchestra.run.smithi112.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-03-27T10:00:52.970 INFO:teuthology.orchestra.run.smithi112.stdout: libtool-2.4.6-25.el8.x86_64 2023-03-27T10:00:52.971 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind-1.3.1-3.el8.x86_64 2023-03-27T10:00:52.971 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-03-27T10:00:52.971 INFO:teuthology.orchestra.run.smithi112.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-03-27T10:00:52.971 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-03-27T10:00:52.971 INFO:teuthology.orchestra.run.smithi112.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-03-27T10:00:52.971 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-0.3.15-3.el8.x86_64 2023-03-27T10:00:52.971 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-03-27T10:00:52.971 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-03-27T10:00:52.972 INFO:teuthology.orchestra.run.smithi112.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-03-27T10:00:52.972 INFO:teuthology.orchestra.run.smithi112.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-03-27T10:00:52.972 INFO:teuthology.orchestra.run.smithi112.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-03-27T10:00:52.972 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-03-27T10:00:52.972 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-03-27T10:00:52.972 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-03-27T10:00:52.972 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-03-27T10:00:52.972 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel-2.5.1-7.el8.noarch 2023-03-27T10:00:52.973 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-03-27T10:00:52.973 INFO:teuthology.orchestra.run.smithi112.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-03-27T10:00:52.973 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-03-27T10:00:52.973 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:00:52.973 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:00:52.973 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:00:52.973 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-03-27T10:00:52.974 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-03-27T10:00:52.974 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-03-27T10:00:52.974 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-03-27T10:00:52.974 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-03-27T10:00:52.974 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-03-27T10:00:52.974 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-6.2-6.el8.noarch 2023-03-27T10:00:52.974 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-03-27T10:00:52.975 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-03-27T10:00:52.975 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-03-27T10:00:52.975 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-03-27T10:00:52.975 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-03-27T10:00:52.975 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-03-27T10:00:52.975 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-03-27T10:00:52.975 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako-1.0.6-13.el8.noarch 2023-03-27T10:00:52.975 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-03-27T10:00:52.976 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-03-27T10:00:52.976 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-03-27T10:00:52.976 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-03-27T10:00:52.976 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-03-27T10:00:52.976 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-03-27T10:00:52.976 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-03-27T10:00:52.976 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend-2.6-1.el8.noarch 2023-03-27T10:00:52.976 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-03-27T10:00:52.977 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-03-27T10:00:52.977 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-03-27T10:00:52.977 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-03-27T10:00:52.977 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser-2.14-14.el8.noarch 2023-03-27T10:00:52.977 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz-2017.2-9.el8.noarch 2023-03-27T10:00:52.977 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:00:52.977 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:00:52.978 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-03-27T10:00:52.978 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-03-27T10:00:52.978 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:00:52.978 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes-2.4.1-12.el8.noarch 2023-03-27T10:00:52.978 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa-4.8-1.el8.noarch 2023-03-27T10:00:52.978 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-03-27T10:00:52.978 INFO:teuthology.orchestra.run.smithi112.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-03-27T10:00:52.978 INFO:teuthology.orchestra.run.smithi112.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-03-27T10:00:52.979 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-03-27T10:00:52.979 INFO:teuthology.orchestra.run.smithi112.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-03-27T10:00:52.979 INFO:teuthology.orchestra.run.smithi112.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-03-27T10:00:52.979 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-03-27T10:00:52.979 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-03-27T10:00:52.980 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-03-27T10:00:52.980 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-03-27T10:00:52.980 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-03-27T10:00:52.980 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:00:52.980 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:00:52.980 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:00:52.980 INFO:teuthology.orchestra.run.smithi112.stdout: socat-1.7.4.1-1.el8.x86_64 2023-03-27T10:00:52.980 INFO:teuthology.orchestra.run.smithi112.stdout: thrift-0.13.0-2.el8.x86_64 2023-03-27T10:00:52.981 INFO:teuthology.orchestra.run.smithi112.stdout: uuid-1.6.2-43.el8.x86_64 2023-03-27T10:00:52.981 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-03-27T10:00:52.981 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-03-27T10:00:52.981 INFO:teuthology.orchestra.run.smithi112.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-03-27T10:00:52.982 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-03-27T10:00:52.982 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-03-27T10:00:53.096 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-03-27T10:00:53.180 DEBUG:teuthology.parallel:result is None 2023-03-27T10:00:53.358 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-03-27T10:00:54.651 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-03-27T10:00:54.956 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-03-27T10:00:55.140 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : ceph-fuse-2:18.0.0-2533.gb70fdf97.el8.x86_64 119/134 2023-03-27T10:00:55.343 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-03-27T10:00:55.535 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : libcephfs-devel-2:18.0.0-2533.gb70fdf97.el8.x86_ 121/134 2023-03-27T10:00:55.697 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-03-27T10:00:58.908 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-03-27T10:00:59.171 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-03-27T10:00:59.205 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-03-27T10:00:59.506 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : cephfs-top-2:18.0.0-2533.gb70fdf97.el8.noarch 124/134 2023-03-27T10:00:59.715 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : rbd-fuse-2:18.0.0-2533.gb70fdf97.el8.x86_64 125/134 2023-03-27T10:01:00.048 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : rbd-nbd-2:18.0.0-2533.gb70fdf97.el8.x86_64 126/134 2023-03-27T10:01:00.348 INFO:teuthology.orchestra.run.smithi023.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-03-27T10:01:00.513 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-03-27T10:01:00.533 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-03-27T10:01:00.760 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-03-27T10:01:00.954 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-03-27T10:01:01.066 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-03-27T10:01:01.320 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-03-27T10:01:01.347 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-03-27T10:01:01.348 INFO:teuthology.orchestra.run.smithi023.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-03-27T10:01:01.569 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-03-27T10:01:01.569 INFO:teuthology.orchestra.run.smithi023.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-03-27T10:01:01.836 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-03-27T10:01:02.012 INFO:teuthology.orchestra.run.smithi023.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-03-27T10:01:02.245 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-03-27T10:01:02.246 INFO:teuthology.orchestra.run.smithi023.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-03-27T10:01:02.246 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:01:02.542 INFO:teuthology.orchestra.run.smithi023.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-03-27T10:01:02.543 INFO:teuthology.orchestra.run.smithi023.stdout:All rules containing unresolvable specifiers will be skipped. 2023-03-27T10:01:02.543 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:01:02.847 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : ceph-2:18.0.0-2533.gb70fdf97.el8.x86_64 1/134 2023-03-27T10:01:02.847 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : ceph-base-2:18.0.0-2533.gb70fdf97.el8.x86_64 2/134 2023-03-27T10:01:02.847 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : ceph-common-2:18.0.0-2533.gb70fdf97.el8.x86_64 3/134 2023-03-27T10:01:02.848 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : ceph-fuse-2:18.0.0-2533.gb70fdf97.el8.x86_64 4/134 2023-03-27T10:01:02.848 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-2533.gb70fd 5/134 2023-03-27T10:01:02.848 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : ceph-mds-2:18.0.0-2533.gb70fdf97.el8.x86_64 6/134 2023-03-27T10:01:02.848 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : ceph-mgr-2:18.0.0-2533.gb70fdf97.el8.x86_64 7/134 2023-03-27T10:01:02.848 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : ceph-mon-2:18.0.0-2533.gb70fdf97.el8.x86_64 8/134 2023-03-27T10:01:02.848 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : ceph-osd-2:18.0.0-2533.gb70fdf97.el8.x86_64 9/134 2023-03-27T10:01:02.848 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : ceph-radosgw-2:18.0.0-2533.gb70fdf97.el8.x86_64 10/134 2023-03-27T10:01:02.848 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : ceph-selinux-2:18.0.0-2533.gb70fdf97.el8.x86_64 11/134 2023-03-27T10:01:02.849 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : ceph-test-2:18.0.0-2533.gb70fdf97.el8.x86_64 12/134 2023-03-27T10:01:02.849 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : cephfs-mirror-2:18.0.0-2533.gb70fdf97.el8.x86_64 13/134 2023-03-27T10:01:02.849 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : libcephfs-devel-2:18.0.0-2533.gb70fdf97.el8.x86_ 14/134 2023-03-27T10:01:02.849 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : libcephfs2-2:18.0.0-2533.gb70fdf97.el8.x86_64 15/134 2023-03-27T10:01:02.849 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : libcephsqlite-2:18.0.0-2533.gb70fdf97.el8.x86_64 16/134 2023-03-27T10:01:02.849 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : librados-devel-2:18.0.0-2533.gb70fdf97.el8.x86_6 17/134 2023-03-27T10:01:02.849 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : libradosstriper1-2:18.0.0-2533.gb70fdf97.el8.x86 18/134 2023-03-27T10:01:02.850 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : librgw2-2:18.0.0-2533.gb70fdf97.el8.x86_64 19/134 2023-03-27T10:01:02.850 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-ceph-argparse-2:18.0.0-2533.gb70fdf97.el 20/134 2023-03-27T10:01:02.850 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-ceph-common-2:18.0.0-2533.gb70fdf97.el8. 21/134 2023-03-27T10:01:02.850 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-cephfs-2:18.0.0-2533.gb70fdf97.el8.x86_6 22/134 2023-03-27T10:01:02.850 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-rados-2:18.0.0-2533.gb70fdf97.el8.x86_64 23/134 2023-03-27T10:01:02.850 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-rbd-2:18.0.0-2533.gb70fdf97.el8.x86_64 24/134 2023-03-27T10:01:02.850 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-rgw-2:18.0.0-2533.gb70fdf97.el8.x86_64 25/134 2023-03-27T10:01:02.850 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : rbd-fuse-2:18.0.0-2533.gb70fdf97.el8.x86_64 26/134 2023-03-27T10:01:02.851 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : rbd-mirror-2:18.0.0-2533.gb70fdf97.el8.x86_64 27/134 2023-03-27T10:01:02.851 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : rbd-nbd-2:18.0.0-2533.gb70fdf97.el8.x86_64 28/134 2023-03-27T10:01:02.851 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-2533.gb70fdf97. 29/134 2023-03-27T10:01:02.851 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-2533.gb70fdf97.el8.noa 30/134 2023-03-27T10:01:02.851 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-2533.gb70fdf97.el8.n 31/134 2023-03-27T10:01:02.851 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-2533.gb70 32/134 2023-03-27T10:01:02.851 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-2533.gb70fdf97.el 33/134 2023-03-27T10:01:02.851 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : ceph-mgr-rook-2:18.0.0-2533.gb70fdf97.el8.noarch 34/134 2023-03-27T10:01:02.852 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-2533.gb70fdf97.e 35/134 2023-03-27T10:01:02.852 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : ceph-volume-2:18.0.0-2533.gb70fdf97.el8.noarch 36/134 2023-03-27T10:01:02.852 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : cephadm-2:18.0.0-2533.gb70fdf97.el8.noarch 37/134 2023-03-27T10:01:02.852 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : cephfs-top-2:18.0.0-2533.gb70fdf97.el8.noarch 38/134 2023-03-27T10:01:02.852 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-03-27T10:01:02.852 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-03-27T10:01:02.852 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-03-27T10:01:02.853 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-03-27T10:01:02.853 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-03-27T10:01:02.853 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-03-27T10:01:02.853 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-03-27T10:01:02.853 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-03-27T10:01:02.853 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-03-27T10:01:02.853 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-03-27T10:01:02.853 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-03-27T10:01:02.854 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-03-27T10:01:02.854 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-03-27T10:01:02.854 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-03-27T10:01:02.854 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-03-27T10:01:02.854 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-03-27T10:01:02.854 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-03-27T10:01:02.854 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-03-27T10:01:02.854 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-03-27T10:01:02.855 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-03-27T10:01:02.855 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-03-27T10:01:02.855 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-03-27T10:01:02.855 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-03-27T10:01:02.855 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-03-27T10:01:02.855 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-03-27T10:01:02.855 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-03-27T10:01:02.855 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 65/134 2023-03-27T10:01:02.856 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-03-27T10:01:02.856 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-03-27T10:01:02.856 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-03-27T10:01:02.856 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-03-27T10:01:02.856 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-03-27T10:01:02.856 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-03-27T10:01:02.856 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-03-27T10:01:02.856 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-03-27T10:01:02.857 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-03-27T10:01:02.857 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-03-27T10:01:02.857 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-03-27T10:01:02.857 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-03-27T10:01:02.857 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-03-27T10:01:02.857 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-03-27T10:01:02.857 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-03-27T10:01:02.858 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-03-27T10:01:02.858 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-03-27T10:01:02.858 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-03-27T10:01:02.858 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-03-27T10:01:02.858 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-03-27T10:01:02.858 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-03-27T10:01:02.858 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-03-27T10:01:02.858 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-03-27T10:01:02.859 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-03-27T10:01:02.859 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-03-27T10:01:02.859 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-03-27T10:01:02.859 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-03-27T10:01:02.859 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-03-27T10:01:02.859 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-03-27T10:01:02.859 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-03-27T10:01:02.859 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-03-27T10:01:02.860 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-03-27T10:01:02.860 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-03-27T10:01:02.860 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-03-27T10:01:02.860 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-03-27T10:01:02.860 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-03-27T10:01:02.860 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-03-27T10:01:02.860 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-03-27T10:01:02.860 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-03-27T10:01:02.861 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-03-27T10:01:02.861 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-03-27T10:01:02.861 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-03-27T10:01:02.861 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-03-27T10:01:02.861 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-03-27T10:01:02.861 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-03-27T10:01:02.861 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-03-27T10:01:02.862 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-03-27T10:01:02.862 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-03-27T10:01:02.862 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-03-27T10:01:02.862 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-03-27T10:01:02.862 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-03-27T10:01:02.862 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-03-27T10:01:02.862 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-03-27T10:01:02.862 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-03-27T10:01:02.863 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-03-27T10:01:02.863 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-03-27T10:01:02.863 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-03-27T10:01:02.863 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-03-27T10:01:02.863 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-03-27T10:01:02.863 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-03-27T10:01:02.863 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-03-27T10:01:02.863 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-03-27T10:01:02.864 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-03-27T10:01:02.864 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : librados2-2:18.0.0-2533.gb70fdf97.el8.x86_64 129/134 2023-03-27T10:01:02.864 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-03-27T10:01:02.864 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : librbd1-2:18.0.0-2533.gb70fdf97.el8.x86_64 131/134 2023-03-27T10:01:02.864 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-03-27T10:01:02.864 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-03-27T10:01:05.642 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-03-27T10:01:05.643 INFO:teuthology.orchestra.run.smithi023.stdout:Last metadata expiration check: 0:02:05 ago on Mon 27 Mar 2023 09:59:00 AM UTC. 2023-03-27T10:01:08.384 INFO:teuthology.orchestra.run.smithi023.stdout:Installed products updated. 2023-03-27T10:01:08.633 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:01:08.634 INFO:teuthology.orchestra.run.smithi023.stdout:Upgraded: 2023-03-27T10:01:08.634 INFO:teuthology.orchestra.run.smithi023.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-03-27T10:01:08.634 INFO:teuthology.orchestra.run.smithi023.stdout: librados2-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:01:08.634 INFO:teuthology.orchestra.run.smithi023.stdout: librbd1-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:01:08.634 INFO:teuthology.orchestra.run.smithi023.stdout:Installed: 2023-03-27T10:01:08.635 INFO:teuthology.orchestra.run.smithi023.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-03-27T10:01:08.635 INFO:teuthology.orchestra.run.smithi023.stdout: automake-1.16.1-7.el8.noarch 2023-03-27T10:01:08.635 INFO:teuthology.orchestra.run.smithi023.stdout: bison-3.0.4-10.el8.x86_64 2023-03-27T10:01:08.635 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:01:08.635 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-base-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:01:08.635 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-common-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:01:08.635 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-fuse-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:01:08.635 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-grafana-dashboards-2:18.0.0-2533.gb70fdf97.el8.noarch 2023-03-27T10:01:08.636 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-immutable-object-cache-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:01:08.636 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-mds-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:01:08.636 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-mgr-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:01:08.636 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-mgr-cephadm-2:18.0.0-2533.gb70fdf97.el8.noarch 2023-03-27T10:01:08.636 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-mgr-dashboard-2:18.0.0-2533.gb70fdf97.el8.noarch 2023-03-27T10:01:08.636 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-mgr-diskprediction-local-2:18.0.0-2533.gb70fdf97.el8.noarch 2023-03-27T10:01:08.636 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-mgr-modules-core-2:18.0.0-2533.gb70fdf97.el8.noarch 2023-03-27T10:01:08.637 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-mgr-rook-2:18.0.0-2533.gb70fdf97.el8.noarch 2023-03-27T10:01:08.637 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-mon-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:01:08.637 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-osd-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:01:08.637 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-prometheus-alerts-2:18.0.0-2533.gb70fdf97.el8.noarch 2023-03-27T10:01:08.637 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-radosgw-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:01:08.637 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-selinux-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:01:08.637 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-test-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:01:08.638 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-volume-2:18.0.0-2533.gb70fdf97.el8.noarch 2023-03-27T10:01:08.638 INFO:teuthology.orchestra.run.smithi023.stdout: cephadm-2:18.0.0-2533.gb70fdf97.el8.noarch 2023-03-27T10:01:08.638 INFO:teuthology.orchestra.run.smithi023.stdout: cephfs-mirror-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:01:08.638 INFO:teuthology.orchestra.run.smithi023.stdout: cephfs-top-2:18.0.0-2533.gb70fdf97.el8.noarch 2023-03-27T10:01:08.638 INFO:teuthology.orchestra.run.smithi023.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-03-27T10:01:08.638 INFO:teuthology.orchestra.run.smithi023.stdout: flex-2.6.1-9.el8.x86_64 2023-03-27T10:01:08.638 INFO:teuthology.orchestra.run.smithi023.stdout: fuse-2.9.7-15.el8.x86_64 2023-03-27T10:01:08.639 INFO:teuthology.orchestra.run.smithi023.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-03-27T10:01:08.639 INFO:teuthology.orchestra.run.smithi023.stdout: jq-1.6-3.el8.x86_64 2023-03-27T10:01:08.639 INFO:teuthology.orchestra.run.smithi023.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-03-27T10:01:08.639 INFO:teuthology.orchestra.run.smithi023.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-03-27T10:01:08.639 INFO:teuthology.orchestra.run.smithi023.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-03-27T10:01:08.639 INFO:teuthology.orchestra.run.smithi023.stdout: libcephfs-devel-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:01:08.639 INFO:teuthology.orchestra.run.smithi023.stdout: libcephfs2-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:01:08.639 INFO:teuthology.orchestra.run.smithi023.stdout: libcephsqlite-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:01:08.640 INFO:teuthology.orchestra.run.smithi023.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-03-27T10:01:08.640 INFO:teuthology.orchestra.run.smithi023.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-03-27T10:01:08.640 INFO:teuthology.orchestra.run.smithi023.stdout: liboath-2.6.2-3.el8.x86_64 2023-03-27T10:01:08.640 INFO:teuthology.orchestra.run.smithi023.stdout: libpq-13.5-1.el8.x86_64 2023-03-27T10:01:08.640 INFO:teuthology.orchestra.run.smithi023.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-03-27T10:01:08.640 INFO:teuthology.orchestra.run.smithi023.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-03-27T10:01:08.640 INFO:teuthology.orchestra.run.smithi023.stdout: librados-devel-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:01:08.640 INFO:teuthology.orchestra.run.smithi023.stdout: libradosstriper1-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:01:08.641 INFO:teuthology.orchestra.run.smithi023.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-03-27T10:01:08.641 INFO:teuthology.orchestra.run.smithi023.stdout: librgw2-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:01:08.641 INFO:teuthology.orchestra.run.smithi023.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-03-27T10:01:08.641 INFO:teuthology.orchestra.run.smithi023.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-03-27T10:01:08.641 INFO:teuthology.orchestra.run.smithi023.stdout: libtool-2.4.6-25.el8.x86_64 2023-03-27T10:01:08.641 INFO:teuthology.orchestra.run.smithi023.stdout: libunwind-1.3.1-3.el8.x86_64 2023-03-27T10:01:08.641 INFO:teuthology.orchestra.run.smithi023.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-03-27T10:01:08.642 INFO:teuthology.orchestra.run.smithi023.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-03-27T10:01:08.642 INFO:teuthology.orchestra.run.smithi023.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-03-27T10:01:08.642 INFO:teuthology.orchestra.run.smithi023.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-03-27T10:01:08.642 INFO:teuthology.orchestra.run.smithi023.stdout: openblas-0.3.15-3.el8.x86_64 2023-03-27T10:01:08.642 INFO:teuthology.orchestra.run.smithi023.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-03-27T10:01:08.642 INFO:teuthology.orchestra.run.smithi023.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-03-27T10:01:08.643 INFO:teuthology.orchestra.run.smithi023.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-03-27T10:01:08.643 INFO:teuthology.orchestra.run.smithi023.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-03-27T10:01:08.643 INFO:teuthology.orchestra.run.smithi023.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-03-27T10:01:08.643 INFO:teuthology.orchestra.run.smithi023.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-03-27T10:01:08.644 INFO:teuthology.orchestra.run.smithi023.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-03-27T10:01:08.644 INFO:teuthology.orchestra.run.smithi023.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-03-27T10:01:08.644 INFO:teuthology.orchestra.run.smithi023.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-03-27T10:01:08.644 INFO:teuthology.orchestra.run.smithi023.stdout: python3-babel-2.5.1-7.el8.noarch 2023-03-27T10:01:08.644 INFO:teuthology.orchestra.run.smithi023.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-03-27T10:01:08.644 INFO:teuthology.orchestra.run.smithi023.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-03-27T10:01:08.644 INFO:teuthology.orchestra.run.smithi023.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-03-27T10:01:08.645 INFO:teuthology.orchestra.run.smithi023.stdout: python3-ceph-argparse-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:01:08.645 INFO:teuthology.orchestra.run.smithi023.stdout: python3-ceph-common-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:01:08.645 INFO:teuthology.orchestra.run.smithi023.stdout: python3-cephfs-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:01:08.645 INFO:teuthology.orchestra.run.smithi023.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-03-27T10:01:08.645 INFO:teuthology.orchestra.run.smithi023.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-03-27T10:01:08.645 INFO:teuthology.orchestra.run.smithi023.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-03-27T10:01:08.645 INFO:teuthology.orchestra.run.smithi023.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-03-27T10:01:08.646 INFO:teuthology.orchestra.run.smithi023.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-03-27T10:01:08.646 INFO:teuthology.orchestra.run.smithi023.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-03-27T10:01:08.646 INFO:teuthology.orchestra.run.smithi023.stdout: python3-jaraco-6.2-6.el8.noarch 2023-03-27T10:01:08.646 INFO:teuthology.orchestra.run.smithi023.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-03-27T10:01:08.646 INFO:teuthology.orchestra.run.smithi023.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-03-27T10:01:08.646 INFO:teuthology.orchestra.run.smithi023.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-03-27T10:01:08.647 INFO:teuthology.orchestra.run.smithi023.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-03-27T10:01:08.647 INFO:teuthology.orchestra.run.smithi023.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-03-27T10:01:08.647 INFO:teuthology.orchestra.run.smithi023.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-03-27T10:01:08.647 INFO:teuthology.orchestra.run.smithi023.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-03-27T10:01:08.647 INFO:teuthology.orchestra.run.smithi023.stdout: python3-mako-1.0.6-13.el8.noarch 2023-03-27T10:01:08.647 INFO:teuthology.orchestra.run.smithi023.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-03-27T10:01:08.647 INFO:teuthology.orchestra.run.smithi023.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-03-27T10:01:08.648 INFO:teuthology.orchestra.run.smithi023.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-03-27T10:01:08.648 INFO:teuthology.orchestra.run.smithi023.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-03-27T10:01:08.648 INFO:teuthology.orchestra.run.smithi023.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-03-27T10:01:08.648 INFO:teuthology.orchestra.run.smithi023.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-03-27T10:01:08.648 INFO:teuthology.orchestra.run.smithi023.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-03-27T10:01:08.648 INFO:teuthology.orchestra.run.smithi023.stdout: python3-portend-2.6-1.el8.noarch 2023-03-27T10:01:08.648 INFO:teuthology.orchestra.run.smithi023.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-03-27T10:01:08.649 INFO:teuthology.orchestra.run.smithi023.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-03-27T10:01:08.649 INFO:teuthology.orchestra.run.smithi023.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-03-27T10:01:08.649 INFO:teuthology.orchestra.run.smithi023.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-03-27T10:01:08.649 INFO:teuthology.orchestra.run.smithi023.stdout: python3-pycparser-2.14-14.el8.noarch 2023-03-27T10:01:08.649 INFO:teuthology.orchestra.run.smithi023.stdout: python3-pytz-2017.2-9.el8.noarch 2023-03-27T10:01:08.649 INFO:teuthology.orchestra.run.smithi023.stdout: python3-rados-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:01:08.650 INFO:teuthology.orchestra.run.smithi023.stdout: python3-rbd-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:01:08.650 INFO:teuthology.orchestra.run.smithi023.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-03-27T10:01:08.650 INFO:teuthology.orchestra.run.smithi023.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-03-27T10:01:08.650 INFO:teuthology.orchestra.run.smithi023.stdout: python3-rgw-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:01:08.650 INFO:teuthology.orchestra.run.smithi023.stdout: python3-routes-2.4.1-12.el8.noarch 2023-03-27T10:01:08.650 INFO:teuthology.orchestra.run.smithi023.stdout: python3-rsa-4.8-1.el8.noarch 2023-03-27T10:01:08.650 INFO:teuthology.orchestra.run.smithi023.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-03-27T10:01:08.650 INFO:teuthology.orchestra.run.smithi023.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-03-27T10:01:08.651 INFO:teuthology.orchestra.run.smithi023.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-03-27T10:01:08.651 INFO:teuthology.orchestra.run.smithi023.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-03-27T10:01:08.651 INFO:teuthology.orchestra.run.smithi023.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-03-27T10:01:08.651 INFO:teuthology.orchestra.run.smithi023.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-03-27T10:01:08.651 INFO:teuthology.orchestra.run.smithi023.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-03-27T10:01:08.651 INFO:teuthology.orchestra.run.smithi023.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-03-27T10:01:08.651 INFO:teuthology.orchestra.run.smithi023.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-03-27T10:01:08.651 INFO:teuthology.orchestra.run.smithi023.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-03-27T10:01:08.652 INFO:teuthology.orchestra.run.smithi023.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-03-27T10:01:08.652 INFO:teuthology.orchestra.run.smithi023.stdout: rbd-fuse-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:01:08.652 INFO:teuthology.orchestra.run.smithi023.stdout: rbd-mirror-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:01:08.652 INFO:teuthology.orchestra.run.smithi023.stdout: rbd-nbd-2:18.0.0-2533.gb70fdf97.el8.x86_64 2023-03-27T10:01:08.652 INFO:teuthology.orchestra.run.smithi023.stdout: socat-1.7.4.1-1.el8.x86_64 2023-03-27T10:01:08.652 INFO:teuthology.orchestra.run.smithi023.stdout: thrift-0.13.0-2.el8.x86_64 2023-03-27T10:01:08.652 INFO:teuthology.orchestra.run.smithi023.stdout: uuid-1.6.2-43.el8.x86_64 2023-03-27T10:01:08.652 INFO:teuthology.orchestra.run.smithi023.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-03-27T10:01:08.653 INFO:teuthology.orchestra.run.smithi023.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-03-27T10:01:08.653 INFO:teuthology.orchestra.run.smithi023.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-03-27T10:01:08.653 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:01:08.654 INFO:teuthology.orchestra.run.smithi023.stdout:Complete! 2023-03-27T10:01:09.864 DEBUG:teuthology.parallel:result is None 2023-03-27T10:01:09.865 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b70fdf973c2ffa23c0d8aa6918519da05d922302 2023-03-27T10:01:10.005 DEBUG:teuthology.orchestra.run.smithi023:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-03-27T10:01:10.056 INFO:teuthology.orchestra.run.smithi023.stdout:18.0.0-2533.gb70fdf97.el8 2023-03-27T10:01:10.057 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-2533.gb70fdf97.el8 2023-03-27T10:01:10.057 INFO:teuthology.task.install:The correct ceph version 18.0.0-2533.gb70fdf97 is installed. 2023-03-27T10:01:10.057 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b70fdf973c2ffa23c0d8aa6918519da05d922302 2023-03-27T10:01:10.194 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-03-27T10:01:10.218 INFO:teuthology.orchestra.run.smithi112.stdout:18.0.0-2533.gb70fdf97.el8 2023-03-27T10:01:10.218 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-2533.gb70fdf97.el8 2023-03-27T10:01:10.219 INFO:teuthology.task.install:The correct ceph version 18.0.0-2533.gb70fdf97 is installed. 2023-03-27T10:01:10.219 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-03-27T10:01:10.220 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2023-03-27T10:01:10.220 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-03-27T10:01:10.303 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-03-27T10:01:10.303 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-03-27T10:01:10.344 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-03-27T10:01:10.344 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2023-03-27T10:01:10.345 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd of=/usr/bin/daemon-helper 2023-03-27T10:01:10.408 DEBUG:teuthology.orchestra.run.smithi023:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-03-27T10:01:10.511 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-03-27T10:01:10.512 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/daemon-helper 2023-03-27T10:01:10.539 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-03-27T10:01:10.604 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-03-27T10:01:10.605 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2023-03-27T10:01:10.605 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd of=/usr/bin/adjust-ulimits 2023-03-27T10:01:10.665 DEBUG:teuthology.orchestra.run.smithi023:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-03-27T10:01:10.764 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-03-27T10:01:10.765 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/adjust-ulimits 2023-03-27T10:01:10.791 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-03-27T10:01:10.858 INFO:teuthology.run_tasks:Running task ceph... 2023-03-27T10:01:10.938 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-03-27T10:01:10.939 DEBUG:teuthology.orchestra.run.smithi023:> sudo chmod 777 /var/log/ceph 2023-03-27T10:01:10.943 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 777 /var/log/ceph 2023-03-27T10:01:11.000 INFO:tasks.ceph:Disabling ceph logrotate... 2023-03-27T10:01:11.001 DEBUG:teuthology.orchestra.run.smithi023:> sudo rm -f -- /etc/logrotate.d/ceph 2023-03-27T10:01:11.005 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /etc/logrotate.d/ceph 2023-03-27T10:01:11.061 INFO:tasks.ceph:Creating extra log directories... 2023-03-27T10:01:11.061 DEBUG:teuthology.orchestra.run.smithi023:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-03-27T10:01:11.065 DEBUG:teuthology.orchestra.run.smithi112:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-03-27T10:01:11.128 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-03-27T10:01:11.129 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-03-27T10:01:11.129 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-03-27T10:01:11.129 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2023-03-27T10:01:11.130 DEBUG:teuthology.orchestra.run.smithi023:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-03-27T10:01:11.167 DEBUG:teuthology.orchestra.run.smithi023:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-03-27T10:01:11.402 DEBUG:teuthology.orchestra.run.smithi023:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-03-27T10:01:11.463 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-03-27T10:01:11.464 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-03-27T10:01:11.480 DEBUG:teuthology.orchestra.run.smithi112:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-03-27T10:01:11.653 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-03-27T10:01:11.685 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-03-27T10:01:11.685 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': False, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'overall HEALTH_', '\\(MON_DOWN\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-03-27T10:01:11.686 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2023-03-27_06:39:30-fs-wip-rishabh-cephman-6-testing-default-smithi/7221293', 'branch': 'wip-rishabh-cephman-6', 'description': 'fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/snaps}}', 'email': None, 'first_in_suite': False, 'job_id': '7221293', 'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '1776e8f494e94fd3414d7679e5782ec22e83c95f'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2023-03-27_06:39:30-fs-wip-rishabh-cephman-6-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-cephman-6'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': False, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'overall HEALTH_', '\\(MON_DOWN\\)'], 'sha1': 'b70fdf973c2ffa23c0d8aa6918519da05d922302'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'check-counter': {'counters': {'mds': ['mds.exported', 'mds.imported']}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'b70fdf973c2ffa23c0d8aa6918519da05d922302'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'CephManager-in-CephFSTestCase', 'sha1': '342392241eddd9827618f3dbb8134df7441718a8'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 200, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'sha1': 'b70fdf973c2ffa23c0d8aa6918519da05d922302', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'CephManager-in-CephFSTestCase', 'suite_path': '/home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/rishabh-d-dave/ceph', 'suite_sha1': '342392241eddd9827618f3dbb8134df7441718a8', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi023.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs=', 'smithi112.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '1776e8f494e94fd3414d7679e5782ec22e83c95f'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b70fdf973c2ffa23c0d8aa6918519da05d922302'}}, {'ceph': None}, {'kclient': None}, {'mon_thrash': {'check_mds_failover': True, 'revive_delay': 20, 'thrash_delay': 10}}, {'workunit': {'clients': {'all': ['fs/snaps']}, 'timeout': '6h'}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e3729161eecf8e6d09a4069acbb883cdcd5e9f50', 'timestamp': '2023-03-27_06:39:30', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.4538'} 2023-03-27T10:01:11.686 DEBUG:teuthology.orchestra.run.smithi023:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-03-27T10:01:11.690 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-03-27T10:01:11.736 DEBUG:teuthology.orchestra.run.smithi023:> sudo install -d -m0777 -- /var/run/ceph 2023-03-27T10:01:11.769 DEBUG:teuthology.orchestra.run.smithi112:> sudo install -d -m0777 -- /var/run/ceph 2023-03-27T10:01:11.827 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2023-03-27T10:01:11.827 DEBUG:teuthology.orchestra.run.smithi023:> dd if=/scratch_devs of=/dev/stdout 2023-03-27T10:01:11.864 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-03-27T10:01:11.865 DEBUG:teuthology.orchestra.run.smithi023:> stat /dev/vg_nvme/lv_1 2023-03-27T10:01:11.947 INFO:teuthology.orchestra.run.smithi023.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-03-27T10:01:11.947 INFO:teuthology.orchestra.run.smithi023.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-27T10:01:11.948 INFO:teuthology.orchestra.run.smithi023.stdout:Device: 5h/5d Inode: 448 Links: 1 2023-03-27T10:01:11.948 INFO:teuthology.orchestra.run.smithi023.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-27T10:01:11.948 INFO:teuthology.orchestra.run.smithi023.stdout:Context: system_u:object_r:device_t:s0 2023-03-27T10:01:11.948 INFO:teuthology.orchestra.run.smithi023.stdout:Access: 2023-03-27 10:01:02.364328226 +0000 2023-03-27T10:01:11.948 INFO:teuthology.orchestra.run.smithi023.stdout:Modify: 2023-03-27 09:58:03.080333241 +0000 2023-03-27T10:01:11.948 INFO:teuthology.orchestra.run.smithi023.stdout:Change: 2023-03-27 09:58:03.080333241 +0000 2023-03-27T10:01:11.948 INFO:teuthology.orchestra.run.smithi023.stdout: Birth: 2023-03-27 09:58:03.054333413 +0000 2023-03-27T10:01:11.949 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-03-27T10:01:12.046 INFO:teuthology.orchestra.run.smithi023.stderr:1+0 records in 2023-03-27T10:01:12.046 INFO:teuthology.orchestra.run.smithi023.stderr:1+0 records out 2023-03-27T10:01:12.046 INFO:teuthology.orchestra.run.smithi023.stderr:512 bytes copied, 0.000405761 s, 1.3 MB/s 2023-03-27T10:01:12.048 DEBUG:teuthology.orchestra.run.smithi023:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-03-27T10:01:12.088 DEBUG:teuthology.orchestra.run.smithi023:> stat /dev/vg_nvme/lv_2 2023-03-27T10:01:12.166 INFO:teuthology.orchestra.run.smithi023.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-03-27T10:01:12.167 INFO:teuthology.orchestra.run.smithi023.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-27T10:01:12.167 INFO:teuthology.orchestra.run.smithi023.stdout:Device: 5h/5d Inode: 454 Links: 1 2023-03-27T10:01:12.167 INFO:teuthology.orchestra.run.smithi023.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-27T10:01:12.167 INFO:teuthology.orchestra.run.smithi023.stdout:Context: system_u:object_r:device_t:s0 2023-03-27T10:01:12.167 INFO:teuthology.orchestra.run.smithi023.stdout:Access: 2023-03-27 10:01:02.364328226 +0000 2023-03-27T10:01:12.167 INFO:teuthology.orchestra.run.smithi023.stdout:Modify: 2023-03-27 09:58:03.857328074 +0000 2023-03-27T10:01:12.168 INFO:teuthology.orchestra.run.smithi023.stdout:Change: 2023-03-27 09:58:03.857328074 +0000 2023-03-27T10:01:12.168 INFO:teuthology.orchestra.run.smithi023.stdout: Birth: 2023-03-27 09:58:03.830328254 +0000 2023-03-27T10:01:12.168 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-03-27T10:01:12.264 INFO:teuthology.orchestra.run.smithi023.stderr:1+0 records in 2023-03-27T10:01:12.265 INFO:teuthology.orchestra.run.smithi023.stderr:1+0 records out 2023-03-27T10:01:12.265 INFO:teuthology.orchestra.run.smithi023.stderr:512 bytes copied, 0.000371026 s, 1.4 MB/s 2023-03-27T10:01:12.267 DEBUG:teuthology.orchestra.run.smithi023:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-03-27T10:01:12.307 DEBUG:teuthology.orchestra.run.smithi023:> stat /dev/vg_nvme/lv_3 2023-03-27T10:01:12.386 INFO:teuthology.orchestra.run.smithi023.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-03-27T10:01:12.387 INFO:teuthology.orchestra.run.smithi023.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-27T10:01:12.387 INFO:teuthology.orchestra.run.smithi023.stdout:Device: 5h/5d Inode: 460 Links: 1 2023-03-27T10:01:12.387 INFO:teuthology.orchestra.run.smithi023.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-27T10:01:12.387 INFO:teuthology.orchestra.run.smithi023.stdout:Context: system_u:object_r:device_t:s0 2023-03-27T10:01:12.387 INFO:teuthology.orchestra.run.smithi023.stdout:Access: 2023-03-27 10:01:02.364328226 +0000 2023-03-27T10:01:12.387 INFO:teuthology.orchestra.run.smithi023.stdout:Modify: 2023-03-27 09:58:04.629322942 +0000 2023-03-27T10:01:12.388 INFO:teuthology.orchestra.run.smithi023.stdout:Change: 2023-03-27 09:58:04.629322942 +0000 2023-03-27T10:01:12.388 INFO:teuthology.orchestra.run.smithi023.stdout: Birth: 2023-03-27 09:58:04.601323128 +0000 2023-03-27T10:01:12.388 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-03-27T10:01:12.483 INFO:teuthology.orchestra.run.smithi023.stderr:1+0 records in 2023-03-27T10:01:12.484 INFO:teuthology.orchestra.run.smithi023.stderr:1+0 records out 2023-03-27T10:01:12.484 INFO:teuthology.orchestra.run.smithi023.stderr:512 bytes copied, 0.000361309 s, 1.4 MB/s 2023-03-27T10:01:12.486 DEBUG:teuthology.orchestra.run.smithi023:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-03-27T10:01:12.527 DEBUG:teuthology.orchestra.run.smithi023:> stat /dev/vg_nvme/lv_4 2023-03-27T10:01:12.606 INFO:teuthology.orchestra.run.smithi023.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-03-27T10:01:12.606 INFO:teuthology.orchestra.run.smithi023.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-27T10:01:12.606 INFO:teuthology.orchestra.run.smithi023.stdout:Device: 5h/5d Inode: 466 Links: 1 2023-03-27T10:01:12.607 INFO:teuthology.orchestra.run.smithi023.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-27T10:01:12.607 INFO:teuthology.orchestra.run.smithi023.stdout:Context: system_u:object_r:device_t:s0 2023-03-27T10:01:12.607 INFO:teuthology.orchestra.run.smithi023.stdout:Access: 2023-03-27 10:01:02.365328220 +0000 2023-03-27T10:01:12.607 INFO:teuthology.orchestra.run.smithi023.stdout:Modify: 2023-03-27 09:58:05.406317775 +0000 2023-03-27T10:01:12.607 INFO:teuthology.orchestra.run.smithi023.stdout:Change: 2023-03-27 09:58:05.406317775 +0000 2023-03-27T10:01:12.607 INFO:teuthology.orchestra.run.smithi023.stdout: Birth: 2023-03-27 09:58:05.378317961 +0000 2023-03-27T10:01:12.608 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-03-27T10:01:12.703 INFO:teuthology.orchestra.run.smithi023.stderr:1+0 records in 2023-03-27T10:01:12.704 INFO:teuthology.orchestra.run.smithi023.stderr:1+0 records out 2023-03-27T10:01:12.704 INFO:teuthology.orchestra.run.smithi023.stderr:512 bytes copied, 0.000423989 s, 1.2 MB/s 2023-03-27T10:01:12.706 DEBUG:teuthology.orchestra.run.smithi023:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-03-27T10:01:12.745 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-03-27T10:01:12.746 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-03-27T10:01:12.746 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/scratch_devs of=/dev/stdout 2023-03-27T10:01:12.762 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-03-27T10:01:12.763 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_1 2023-03-27T10:01:12.819 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-03-27T10:01:12.820 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-27T10:01:12.820 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 6h/6d Inode: 360864 Links: 1 2023-03-27T10:01:12.821 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-27T10:01:12.821 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2023-03-27T10:01:12.821 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2023-03-27 10:00:47.074523239 +0000 2023-03-27T10:01:12.821 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2023-03-27 09:55:40.623806284 +0000 2023-03-27T10:01:12.822 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2023-03-27 09:55:40.623806284 +0000 2023-03-27T10:01:12.822 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: - 2023-03-27T10:01:12.822 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-03-27T10:01:12.890 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2023-03-27T10:01:12.890 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2023-03-27T10:01:12.891 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000226597 s, 2.3 MB/s 2023-03-27T10:01:12.892 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-03-27T10:01:12.950 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_2 2023-03-27T10:01:13.006 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-03-27T10:01:13.006 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-27T10:01:13.006 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 6h/6d Inode: 367023 Links: 1 2023-03-27T10:01:13.006 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-27T10:01:13.007 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2023-03-27T10:01:13.007 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2023-03-27 10:00:47.075523213 +0000 2023-03-27T10:01:13.007 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2023-03-27 09:55:41.150792296 +0000 2023-03-27T10:01:13.007 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2023-03-27 09:55:41.150792296 +0000 2023-03-27T10:01:13.007 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: - 2023-03-27T10:01:13.007 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-03-27T10:01:13.072 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2023-03-27T10:01:13.072 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2023-03-27T10:01:13.073 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.00020157 s, 2.5 MB/s 2023-03-27T10:01:13.074 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-03-27T10:01:13.131 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_3 2023-03-27T10:01:13.188 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-03-27T10:01:13.188 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-27T10:01:13.188 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 6h/6d Inode: 363029 Links: 1 2023-03-27T10:01:13.188 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-27T10:01:13.188 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2023-03-27T10:01:13.189 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2023-03-27 10:00:47.075523213 +0000 2023-03-27T10:01:13.189 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2023-03-27 09:55:41.666778600 +0000 2023-03-27T10:01:13.189 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2023-03-27 09:55:41.666778600 +0000 2023-03-27T10:01:13.189 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: - 2023-03-27T10:01:13.189 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-03-27T10:01:13.252 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2023-03-27T10:01:13.252 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2023-03-27T10:01:13.252 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000217726 s, 2.4 MB/s 2023-03-27T10:01:13.253 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-03-27T10:01:13.311 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_4 2023-03-27T10:01:13.366 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-03-27T10:01:13.367 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-27T10:01:13.367 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 6h/6d Inode: 364923 Links: 1 2023-03-27T10:01:13.367 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-27T10:01:13.367 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2023-03-27T10:01:13.367 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2023-03-27 10:00:47.075523213 +0000 2023-03-27T10:01:13.368 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2023-03-27 09:55:42.169765249 +0000 2023-03-27T10:01:13.368 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2023-03-27 09:55:42.169765249 +0000 2023-03-27T10:01:13.368 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: - 2023-03-27T10:01:13.368 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-03-27T10:01:13.431 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2023-03-27T10:01:13.431 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2023-03-27T10:01:13.432 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000184204 s, 2.8 MB/s 2023-03-27T10:01:13.433 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-03-27T10:01:13.490 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-03-27T10:01:13.490 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi023.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi112.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-03-27T10:01:13.490 INFO:tasks.ceph:Generating config... 2023-03-27T10:01:13.491 INFO:tasks.ceph:[client] client mount timeout = 600 2023-03-27T10:01:13.492 INFO:tasks.ceph:[client] debug client = 20 2023-03-27T10:01:13.492 INFO:tasks.ceph:[client] debug ms = 1 2023-03-27T10:01:13.492 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-03-27T10:01:13.492 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-03-27T10:01:13.493 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-03-27T10:01:13.493 INFO:tasks.ceph:[mds] debug mds = 20 2023-03-27T10:01:13.493 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-03-27T10:01:13.493 INFO:tasks.ceph:[mds] debug ms = 1 2023-03-27T10:01:13.494 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2023-03-27T10:01:13.494 INFO:tasks.ceph:[mds] mds bal merge size = 5 2023-03-27T10:01:13.494 INFO:tasks.ceph:[mds] mds bal split bits = 3 2023-03-27T10:01:13.494 INFO:tasks.ceph:[mds] mds bal split size = 100 2023-03-27T10:01:13.495 INFO:tasks.ceph:[mds] mds debug frag = True 2023-03-27T10:01:13.495 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-03-27T10:01:13.495 INFO:tasks.ceph:[mds] mds oft prefetch dirfrags = False 2023-03-27T10:01:13.495 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-03-27T10:01:13.495 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-03-27T10:01:13.496 INFO:tasks.ceph:[mds] mds_dir_prefetch = True 2023-03-27T10:01:13.496 INFO:tasks.ceph:[mds] mds_sleep_rank_change = 5000000.0 2023-03-27T10:01:13.496 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-03-27T10:01:13.496 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-03-27T10:01:13.497 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-03-27T10:01:13.497 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-03-27T10:01:13.497 INFO:tasks.ceph:[mgr] debug ms = 1 2023-03-27T10:01:13.498 INFO:tasks.ceph:[mon] debug mon = 20 2023-03-27T10:01:13.498 INFO:tasks.ceph:[mon] debug ms = 1 2023-03-27T10:01:13.499 INFO:tasks.ceph:[mon] debug paxos = 20 2023-03-27T10:01:13.499 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-03-27T10:01:13.500 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-03-27T10:01:13.500 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2023-03-27T10:01:13.500 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-03-27T10:01:13.500 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-03-27T10:01:13.501 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-03-27T10:01:13.501 INFO:tasks.ceph:[osd] debug ms = 1 2023-03-27T10:01:13.501 INFO:tasks.ceph:[osd] debug osd = 20 2023-03-27T10:01:13.501 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-03-27T10:01:13.501 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-03-27T10:01:13.502 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-03-27T10:01:13.502 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-03-27T10:01:13.502 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2023-03-27T10:01:13.502 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-03-27T10:01:13.502 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2023-03-27T10:01:13.503 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-03-27T10:01:13.503 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-03-27T10:01:13.503 INFO:tasks.ceph:Setting up mon.a... 2023-03-27T10:01:13.503 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-03-27T10:01:13.609 INFO:teuthology.orchestra.run.smithi023.stdout:creating /etc/ceph/ceph.keyring 2023-03-27T10:01:13.615 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-03-27T10:01:13.706 DEBUG:teuthology.orchestra.run.smithi023:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-03-27T10:01:13.766 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.23'), ('mon.b', '172.21.15.112'), ('mon.c', '[v2:172.21.15.112:3301,v1:172.21.15.112:6790]')] 2023-03-27T10:01:13.767 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.23,172.21.15.112,[v2:172.21.15.112:3301,v1:172.21.15.112:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.e': {}, 'mds.b': {}, 'mds.d': {}, 'mds.f': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': False, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-03-27T10:01:13.768 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2023-03-27T10:01:13.768 DEBUG:teuthology.orchestra.run.smithi023:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-03-27T10:01:13.805 DEBUG:teuthology.orchestra.run.smithi023:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.23 --add b 172.21.15.112 --addv c '[v2:172.21.15.112:3301,v1:172.21.15.112:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-03-27T10:01:13.925 INFO:teuthology.orchestra.run.smithi023.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-03-27T10:01:13.925 INFO:teuthology.orchestra.run.smithi023.stdout:monmaptool: generated fsid 1d8d6e6e-0d21-4f51-91ee-d9434d4fb750 2023-03-27T10:01:13.925 INFO:teuthology.orchestra.run.smithi023.stdout:setting min_mon_release = pacific 2023-03-27T10:01:13.925 INFO:teuthology.orchestra.run.smithi023.stdout:epoch 0 2023-03-27T10:01:13.925 INFO:teuthology.orchestra.run.smithi023.stdout:fsid 1d8d6e6e-0d21-4f51-91ee-d9434d4fb750 2023-03-27T10:01:13.926 INFO:teuthology.orchestra.run.smithi023.stdout:last_changed 2023-03-27T10:01:13.926897+0000 2023-03-27T10:01:13.926 INFO:teuthology.orchestra.run.smithi023.stdout:created 2023-03-27T10:01:13.926897+0000 2023-03-27T10:01:13.926 INFO:teuthology.orchestra.run.smithi023.stdout:min_mon_release 16 (pacific) 2023-03-27T10:01:13.926 INFO:teuthology.orchestra.run.smithi023.stdout:election_strategy: 1 2023-03-27T10:01:13.926 INFO:teuthology.orchestra.run.smithi023.stdout:0: [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] mon.a 2023-03-27T10:01:13.927 INFO:teuthology.orchestra.run.smithi023.stdout:1: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.b 2023-03-27T10:01:13.927 INFO:teuthology.orchestra.run.smithi023.stdout:2: [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] mon.c 2023-03-27T10:01:13.927 INFO:teuthology.orchestra.run.smithi023.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-03-27T10:01:13.929 DEBUG:teuthology.orchestra.run.smithi023:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-03-27T10:01:13.968 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 1d8d6e6e-0d21-4f51-91ee-d9434d4fb750... 2023-03-27T10:01:13.970 DEBUG:teuthology.orchestra.run.smithi023:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-03-27T10:01:14.012 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-03-27T10:01:14.056 INFO:teuthology.orchestra.run.smithi112.stdout:[global] 2023-03-27T10:01:14.057 INFO:teuthology.orchestra.run.smithi112.stdout: chdir = "" 2023-03-27T10:01:14.057 INFO:teuthology.orchestra.run.smithi112.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-03-27T10:01:14.057 INFO:teuthology.orchestra.run.smithi112.stdout: auth supported = cephx 2023-03-27T10:01:14.057 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-03-27T10:01:14.057 INFO:teuthology.orchestra.run.smithi112.stdout: filestore xattr use omap = true 2023-03-27T10:01:14.057 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-03-27T10:01:14.058 INFO:teuthology.orchestra.run.smithi112.stdout: mon clock drift allowed = 1.000 2023-03-27T10:01:14.058 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-03-27T10:01:14.058 INFO:teuthology.orchestra.run.smithi112.stdout: osd crush chooseleaf type = 0 2023-03-27T10:01:14.058 INFO:teuthology.orchestra.run.smithi112.stdout: auth debug = true 2023-03-27T10:01:14.058 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-03-27T10:01:14.058 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on old message = true 2023-03-27T10:01:14.058 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on bug = true 2023-03-27T10:01:14.058 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-03-27T10:01:14.059 INFO:teuthology.orchestra.run.smithi112.stdout: mon max pg per osd = 10000 # >= luminous 2023-03-27T10:01:14.059 INFO:teuthology.orchestra.run.smithi112.stdout: mon pg warn max object skew = 0 2023-03-27T10:01:14.059 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-03-27T10:01:14.059 INFO:teuthology.orchestra.run.smithi112.stdout: # disable pg_autoscaler by default for new pools 2023-03-27T10:01:14.059 INFO:teuthology.orchestra.run.smithi112.stdout: osd_pool_default_pg_autoscale_mode = off 2023-03-27T10:01:14.059 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-03-27T10:01:14.059 INFO:teuthology.orchestra.run.smithi112.stdout: osd pool default size = 2 2023-03-27T10:01:14.060 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-03-27T10:01:14.060 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd allow primary affinity = true 2023-03-27T10:01:14.060 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd allow pg remap = true 2023-03-27T10:01:14.060 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on legacy crush tunables = false 2023-03-27T10:01:14.060 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on crush straw calc version zero = false 2023-03-27T10:01:14.060 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on no sortbitwise = false 2023-03-27T10:01:14.060 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on osd down out interval zero = false 2023-03-27T10:01:14.060 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on too few osds = false 2023-03-27T10:01:14.061 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-03-27T10:01:14.061 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_pool_no_redundancy = false 2023-03-27T10:01:14.061 INFO:teuthology.orchestra.run.smithi112.stdout: mon_allow_pool_size_one = true 2023-03-27T10:01:14.061 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-03-27T10:01:14.061 INFO:teuthology.orchestra.run.smithi112.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-03-27T10:01:14.061 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-03-27T10:01:14.061 INFO:teuthology.orchestra.run.smithi112.stdout: osd default data pool replay window = 5 2023-03-27T10:01:14.062 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-03-27T10:01:14.062 INFO:teuthology.orchestra.run.smithi112.stdout: mon allow pool delete = true 2023-03-27T10:01:14.062 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-03-27T10:01:14.062 INFO:teuthology.orchestra.run.smithi112.stdout: mon cluster log file level = debug 2023-03-27T10:01:14.062 INFO:teuthology.orchestra.run.smithi112.stdout: debug asserts on shutdown = true 2023-03-27T10:01:14.062 INFO:teuthology.orchestra.run.smithi112.stdout: mon health detail to clog = false 2023-03-27T10:01:14.062 INFO:teuthology.orchestra.run.smithi112.stdout: mon host = "172.21.15.23,172.21.15.112,[v2:172.21.15.112:3301,v1:172.21.15.112:6790]" 2023-03-27T10:01:14.062 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on skipped message = False 2023-03-27T10:01:14.063 INFO:teuthology.orchestra.run.smithi112.stdout: fsid = 1d8d6e6e-0d21-4f51-91ee-d9434d4fb750 2023-03-27T10:01:14.063 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-03-27T10:01:14.063 INFO:teuthology.orchestra.run.smithi112.stdout:[osd] 2023-03-27T10:01:14.063 INFO:teuthology.orchestra.run.smithi112.stdout: osd journal size = 100 2023-03-27T10:01:14.063 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-03-27T10:01:14.063 INFO:teuthology.orchestra.run.smithi112.stdout: osd scrub load threshold = 5.0 2023-03-27T10:01:14.063 INFO:teuthology.orchestra.run.smithi112.stdout: osd scrub max interval = 600 2023-03-27T10:01:14.063 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-03-27T10:01:14.064 INFO:teuthology.orchestra.run.smithi112.stdout: osd recover clone overlap = true 2023-03-27T10:01:14.064 INFO:teuthology.orchestra.run.smithi112.stdout: osd recovery max chunk = 1048576 2023-03-27T10:01:14.064 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-03-27T10:01:14.064 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug shutdown = true 2023-03-27T10:01:14.064 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug op order = true 2023-03-27T10:01:14.064 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug verify stray on activate = true 2023-03-27T10:01:14.064 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-03-27T10:01:14.064 INFO:teuthology.orchestra.run.smithi112.stdout: osd open classes on start = true 2023-03-27T10:01:14.065 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug pg log writeout = true 2023-03-27T10:01:14.065 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-03-27T10:01:14.065 INFO:teuthology.orchestra.run.smithi112.stdout: osd deep scrub update digest min age = 30 2023-03-27T10:01:14.065 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-03-27T10:01:14.065 INFO:teuthology.orchestra.run.smithi112.stdout: osd map max advance = 10 2023-03-27T10:01:14.065 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-03-27T10:01:14.065 INFO:teuthology.orchestra.run.smithi112.stdout: journal zero on create = true 2023-03-27T10:01:14.065 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-03-27T10:01:14.066 INFO:teuthology.orchestra.run.smithi112.stdout: filestore ondisk finisher threads = 3 2023-03-27T10:01:14.066 INFO:teuthology.orchestra.run.smithi112.stdout: filestore apply finisher threads = 3 2023-03-27T10:01:14.066 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-03-27T10:01:14.066 INFO:teuthology.orchestra.run.smithi112.stdout: bdev debug aio = true 2023-03-27T10:01:14.066 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug misdirected ops = true 2023-03-27T10:01:14.066 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore block size = 96636764160 2023-03-27T10:01:14.066 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore compression mode = aggressive 2023-03-27T10:01:14.066 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore fsck on mount = True 2023-03-27T10:01:14.067 INFO:teuthology.orchestra.run.smithi112.stdout: debug bluefs = 20 2023-03-27T10:01:14.067 INFO:teuthology.orchestra.run.smithi112.stdout: debug bluestore = 20 2023-03-27T10:01:14.067 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2023-03-27T10:01:14.067 INFO:teuthology.orchestra.run.smithi112.stdout: debug osd = 20 2023-03-27T10:01:14.067 INFO:teuthology.orchestra.run.smithi112.stdout: debug rocksdb = 10 2023-03-27T10:01:14.067 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd backfillfull_ratio = 0.85 2023-03-27T10:01:14.067 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd full ratio = 0.9 2023-03-27T10:01:14.068 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd nearfull ratio = 0.8 2023-03-27T10:01:14.068 INFO:teuthology.orchestra.run.smithi112.stdout: osd blocked scrub grace period = 3600 2023-03-27T10:01:14.068 INFO:teuthology.orchestra.run.smithi112.stdout: osd failsafe full ratio = 0.95 2023-03-27T10:01:14.068 INFO:teuthology.orchestra.run.smithi112.stdout: osd max markdown count = 1000 2023-03-27T10:01:14.068 INFO:teuthology.orchestra.run.smithi112.stdout: osd objectstore = bluestore 2023-03-27T10:01:14.068 INFO:teuthology.orchestra.run.smithi112.stdout: osd op complaint time = 180 2023-03-27T10:01:14.068 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-03-27T10:01:14.068 INFO:teuthology.orchestra.run.smithi112.stdout:[mgr] 2023-03-27T10:01:14.069 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2023-03-27T10:01:14.069 INFO:teuthology.orchestra.run.smithi112.stdout: debug mgr = 20 2023-03-27T10:01:14.069 INFO:teuthology.orchestra.run.smithi112.stdout: debug mon = 20 2023-03-27T10:01:14.069 INFO:teuthology.orchestra.run.smithi112.stdout: debug auth = 20 2023-03-27T10:01:14.069 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min pgs per osd = 4 2023-03-27T10:01:14.069 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min bytes per osd = 10 2023-03-27T10:01:14.069 INFO:teuthology.orchestra.run.smithi112.stdout: mgr/telemetry/nag = false 2023-03-27T10:01:14.069 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-03-27T10:01:14.070 INFO:teuthology.orchestra.run.smithi112.stdout:[mon] 2023-03-27T10:01:14.070 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2023-03-27T10:01:14.070 INFO:teuthology.orchestra.run.smithi112.stdout: debug mon = 20 2023-03-27T10:01:14.070 INFO:teuthology.orchestra.run.smithi112.stdout: debug paxos = 20 2023-03-27T10:01:14.070 INFO:teuthology.orchestra.run.smithi112.stdout: debug auth = 20 2023-03-27T10:01:14.070 INFO:teuthology.orchestra.run.smithi112.stdout: mon data avail warn = 5 2023-03-27T10:01:14.070 INFO:teuthology.orchestra.run.smithi112.stdout: mon mgr mkfs grace = 240 2023-03-27T10:01:14.070 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min pgs per osd = 4 2023-03-27T10:01:14.071 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd reporter subtree level = osd 2023-03-27T10:01:14.071 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd prime pg temp = true 2023-03-27T10:01:14.071 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min bytes per osd = 10 2023-03-27T10:01:14.071 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-03-27T10:01:14.071 INFO:teuthology.orchestra.run.smithi112.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-03-27T10:01:14.071 INFO:teuthology.orchestra.run.smithi112.stdout: auth mon ticket ttl = 660 # 11m 2023-03-27T10:01:14.071 INFO:teuthology.orchestra.run.smithi112.stdout: auth service ticket ttl = 240 # 4m 2023-03-27T10:01:14.071 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-03-27T10:01:14.072 INFO:teuthology.orchestra.run.smithi112.stdout: # don't complain about insecure global_id in the test suite 2023-03-27T10:01:14.072 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-03-27T10:01:14.072 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-03-27T10:01:14.072 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-03-27T10:01:14.072 INFO:teuthology.orchestra.run.smithi112.stdout: # 1m isn't quite enough 2023-03-27T10:01:14.072 INFO:teuthology.orchestra.run.smithi112.stdout: mon_down_mkfs_grace = 2m 2023-03-27T10:01:14.072 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-03-27T10:01:14.072 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_filestore_osds = false 2023-03-27T10:01:14.073 INFO:teuthology.orchestra.run.smithi112.stdout: mon op complaint time = 120 2023-03-27T10:01:14.073 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-03-27T10:01:14.073 INFO:teuthology.orchestra.run.smithi112.stdout:[client] 2023-03-27T10:01:14.073 INFO:teuthology.orchestra.run.smithi112.stdout: rgw cache enabled = true 2023-03-27T10:01:14.073 INFO:teuthology.orchestra.run.smithi112.stdout: rgw enable ops log = true 2023-03-27T10:01:14.073 INFO:teuthology.orchestra.run.smithi112.stdout: rgw enable usage log = true 2023-03-27T10:01:14.073 INFO:teuthology.orchestra.run.smithi112.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-03-27T10:01:14.074 INFO:teuthology.orchestra.run.smithi112.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-03-27T10:01:14.074 INFO:teuthology.orchestra.run.smithi112.stdout: client mount timeout = 600 2023-03-27T10:01:14.074 INFO:teuthology.orchestra.run.smithi112.stdout: debug client = 20 2023-03-27T10:01:14.074 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2023-03-27T10:01:14.074 INFO:teuthology.orchestra.run.smithi112.stdout: rados mon op timeout = 900 2023-03-27T10:01:14.074 INFO:teuthology.orchestra.run.smithi112.stdout: rados osd op timeout = 900 2023-03-27T10:01:14.074 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.a] 2023-03-27T10:01:14.074 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.b] 2023-03-27T10:01:14.075 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.c] 2023-03-27T10:01:14.075 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.a] 2023-03-27T10:01:14.075 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.c] 2023-03-27T10:01:14.075 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.e] 2023-03-27T10:01:14.075 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.b] 2023-03-27T10:01:14.076 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.d] 2023-03-27T10:01:14.076 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.f] 2023-03-27T10:01:14.076 INFO:teuthology.orchestra.run.smithi112.stdout:[mds] 2023-03-27T10:01:14.076 INFO:teuthology.orchestra.run.smithi112.stdout: debug mds = 20 2023-03-27T10:01:14.076 INFO:teuthology.orchestra.run.smithi112.stdout: debug mds balancer = 20 2023-03-27T10:01:14.076 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2023-03-27T10:01:14.076 INFO:teuthology.orchestra.run.smithi112.stdout: mds bal fragment size max = 10000 2023-03-27T10:01:14.076 INFO:teuthology.orchestra.run.smithi112.stdout: mds bal merge size = 5 2023-03-27T10:01:14.077 INFO:teuthology.orchestra.run.smithi112.stdout: mds bal split bits = 3 2023-03-27T10:01:14.077 INFO:teuthology.orchestra.run.smithi112.stdout: mds bal split size = 100 2023-03-27T10:01:14.077 INFO:teuthology.orchestra.run.smithi112.stdout: mds debug frag = True 2023-03-27T10:01:14.077 INFO:teuthology.orchestra.run.smithi112.stdout: mds debug scatterstat = True 2023-03-27T10:01:14.077 INFO:teuthology.orchestra.run.smithi112.stdout: mds oft prefetch dirfrags = False 2023-03-27T10:01:14.077 INFO:teuthology.orchestra.run.smithi112.stdout: mds op complaint time = 180 2023-03-27T10:01:14.077 INFO:teuthology.orchestra.run.smithi112.stdout: mds verify scatter = True 2023-03-27T10:01:14.078 INFO:teuthology.orchestra.run.smithi112.stdout: mds_dir_prefetch = True 2023-03-27T10:01:14.078 INFO:teuthology.orchestra.run.smithi112.stdout: mds_sleep_rank_change = 5000000.0 2023-03-27T10:01:14.078 INFO:teuthology.orchestra.run.smithi112.stdout: osd op complaint time = 180 2023-03-27T10:01:14.078 INFO:teuthology.orchestra.run.smithi112.stdout: rados mon op timeout = 900 2023-03-27T10:01:14.078 INFO:teuthology.orchestra.run.smithi112.stdout: rados osd op timeout = 900 2023-03-27T10:01:14.114 INFO:teuthology.orchestra.run.smithi023.stdout:[global] 2023-03-27T10:01:14.114 INFO:teuthology.orchestra.run.smithi023.stdout: chdir = "" 2023-03-27T10:01:14.114 INFO:teuthology.orchestra.run.smithi023.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-03-27T10:01:14.115 INFO:teuthology.orchestra.run.smithi023.stdout: auth supported = cephx 2023-03-27T10:01:14.115 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:01:14.115 INFO:teuthology.orchestra.run.smithi023.stdout: filestore xattr use omap = true 2023-03-27T10:01:14.115 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:01:14.115 INFO:teuthology.orchestra.run.smithi023.stdout: mon clock drift allowed = 1.000 2023-03-27T10:01:14.115 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:01:14.115 INFO:teuthology.orchestra.run.smithi023.stdout: osd crush chooseleaf type = 0 2023-03-27T10:01:14.116 INFO:teuthology.orchestra.run.smithi023.stdout: auth debug = true 2023-03-27T10:01:14.116 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:01:14.116 INFO:teuthology.orchestra.run.smithi023.stdout: ms die on old message = true 2023-03-27T10:01:14.117 INFO:teuthology.orchestra.run.smithi023.stdout: ms die on bug = true 2023-03-27T10:01:14.117 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:01:14.117 INFO:teuthology.orchestra.run.smithi023.stdout: mon max pg per osd = 10000 # >= luminous 2023-03-27T10:01:14.117 INFO:teuthology.orchestra.run.smithi023.stdout: mon pg warn max object skew = 0 2023-03-27T10:01:14.117 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:01:14.117 INFO:teuthology.orchestra.run.smithi023.stdout: # disable pg_autoscaler by default for new pools 2023-03-27T10:01:14.117 INFO:teuthology.orchestra.run.smithi023.stdout: osd_pool_default_pg_autoscale_mode = off 2023-03-27T10:01:14.118 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:01:14.118 INFO:teuthology.orchestra.run.smithi023.stdout: osd pool default size = 2 2023-03-27T10:01:14.118 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:01:14.118 INFO:teuthology.orchestra.run.smithi023.stdout: mon osd allow primary affinity = true 2023-03-27T10:01:14.118 INFO:teuthology.orchestra.run.smithi023.stdout: mon osd allow pg remap = true 2023-03-27T10:01:14.118 INFO:teuthology.orchestra.run.smithi023.stdout: mon warn on legacy crush tunables = false 2023-03-27T10:01:14.118 INFO:teuthology.orchestra.run.smithi023.stdout: mon warn on crush straw calc version zero = false 2023-03-27T10:01:14.119 INFO:teuthology.orchestra.run.smithi023.stdout: mon warn on no sortbitwise = false 2023-03-27T10:01:14.119 INFO:teuthology.orchestra.run.smithi023.stdout: mon warn on osd down out interval zero = false 2023-03-27T10:01:14.119 INFO:teuthology.orchestra.run.smithi023.stdout: mon warn on too few osds = false 2023-03-27T10:01:14.119 INFO:teuthology.orchestra.run.smithi023.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-03-27T10:01:14.119 INFO:teuthology.orchestra.run.smithi023.stdout: mon_warn_on_pool_no_redundancy = false 2023-03-27T10:01:14.119 INFO:teuthology.orchestra.run.smithi023.stdout: mon_allow_pool_size_one = true 2023-03-27T10:01:14.119 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:01:14.119 INFO:teuthology.orchestra.run.smithi023.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-03-27T10:01:14.120 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:01:14.120 INFO:teuthology.orchestra.run.smithi023.stdout: osd default data pool replay window = 5 2023-03-27T10:01:14.120 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:01:14.120 INFO:teuthology.orchestra.run.smithi023.stdout: mon allow pool delete = true 2023-03-27T10:01:14.120 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:01:14.120 INFO:teuthology.orchestra.run.smithi023.stdout: mon cluster log file level = debug 2023-03-27T10:01:14.120 INFO:teuthology.orchestra.run.smithi023.stdout: debug asserts on shutdown = true 2023-03-27T10:01:14.120 INFO:teuthology.orchestra.run.smithi023.stdout: mon health detail to clog = false 2023-03-27T10:01:14.121 INFO:teuthology.orchestra.run.smithi023.stdout: mon host = "172.21.15.23,172.21.15.112,[v2:172.21.15.112:3301,v1:172.21.15.112:6790]" 2023-03-27T10:01:14.121 INFO:teuthology.orchestra.run.smithi023.stdout: ms die on skipped message = False 2023-03-27T10:01:14.121 INFO:teuthology.orchestra.run.smithi023.stdout: fsid = 1d8d6e6e-0d21-4f51-91ee-d9434d4fb750 2023-03-27T10:01:14.121 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:01:14.121 INFO:teuthology.orchestra.run.smithi023.stdout:[osd] 2023-03-27T10:01:14.121 INFO:teuthology.orchestra.run.smithi023.stdout: osd journal size = 100 2023-03-27T10:01:14.121 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:01:14.122 INFO:teuthology.orchestra.run.smithi023.stdout: osd scrub load threshold = 5.0 2023-03-27T10:01:14.122 INFO:teuthology.orchestra.run.smithi023.stdout: osd scrub max interval = 600 2023-03-27T10:01:14.122 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:01:14.122 INFO:teuthology.orchestra.run.smithi023.stdout: osd recover clone overlap = true 2023-03-27T10:01:14.122 INFO:teuthology.orchestra.run.smithi023.stdout: osd recovery max chunk = 1048576 2023-03-27T10:01:14.122 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:01:14.122 INFO:teuthology.orchestra.run.smithi023.stdout: osd debug shutdown = true 2023-03-27T10:01:14.122 INFO:teuthology.orchestra.run.smithi023.stdout: osd debug op order = true 2023-03-27T10:01:14.123 INFO:teuthology.orchestra.run.smithi023.stdout: osd debug verify stray on activate = true 2023-03-27T10:01:14.123 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:01:14.123 INFO:teuthology.orchestra.run.smithi023.stdout: osd open classes on start = true 2023-03-27T10:01:14.123 INFO:teuthology.orchestra.run.smithi023.stdout: osd debug pg log writeout = true 2023-03-27T10:01:14.123 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:01:14.123 INFO:teuthology.orchestra.run.smithi023.stdout: osd deep scrub update digest min age = 30 2023-03-27T10:01:14.123 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:01:14.123 INFO:teuthology.orchestra.run.smithi023.stdout: osd map max advance = 10 2023-03-27T10:01:14.124 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:01:14.124 INFO:teuthology.orchestra.run.smithi023.stdout: journal zero on create = true 2023-03-27T10:01:14.124 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:01:14.124 INFO:teuthology.orchestra.run.smithi023.stdout: filestore ondisk finisher threads = 3 2023-03-27T10:01:14.124 INFO:teuthology.orchestra.run.smithi023.stdout: filestore apply finisher threads = 3 2023-03-27T10:01:14.124 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:01:14.124 INFO:teuthology.orchestra.run.smithi023.stdout: bdev debug aio = true 2023-03-27T10:01:14.124 INFO:teuthology.orchestra.run.smithi023.stdout: osd debug misdirected ops = true 2023-03-27T10:01:14.125 INFO:teuthology.orchestra.run.smithi023.stdout: bluestore block size = 96636764160 2023-03-27T10:01:14.125 INFO:teuthology.orchestra.run.smithi023.stdout: bluestore compression mode = aggressive 2023-03-27T10:01:14.125 INFO:teuthology.orchestra.run.smithi023.stdout: bluestore fsck on mount = True 2023-03-27T10:01:14.125 INFO:teuthology.orchestra.run.smithi023.stdout: debug bluefs = 20 2023-03-27T10:01:14.125 INFO:teuthology.orchestra.run.smithi023.stdout: debug bluestore = 20 2023-03-27T10:01:14.125 INFO:teuthology.orchestra.run.smithi023.stdout: debug ms = 1 2023-03-27T10:01:14.125 INFO:teuthology.orchestra.run.smithi023.stdout: debug osd = 20 2023-03-27T10:01:14.126 INFO:teuthology.orchestra.run.smithi023.stdout: debug rocksdb = 10 2023-03-27T10:01:14.126 INFO:teuthology.orchestra.run.smithi023.stdout: mon osd backfillfull_ratio = 0.85 2023-03-27T10:01:14.126 INFO:teuthology.orchestra.run.smithi023.stdout: mon osd full ratio = 0.9 2023-03-27T10:01:14.126 INFO:teuthology.orchestra.run.smithi023.stdout: mon osd nearfull ratio = 0.8 2023-03-27T10:01:14.126 INFO:teuthology.orchestra.run.smithi023.stdout: osd blocked scrub grace period = 3600 2023-03-27T10:01:14.126 INFO:teuthology.orchestra.run.smithi023.stdout: osd failsafe full ratio = 0.95 2023-03-27T10:01:14.126 INFO:teuthology.orchestra.run.smithi023.stdout: osd max markdown count = 1000 2023-03-27T10:01:14.126 INFO:teuthology.orchestra.run.smithi023.stdout: osd objectstore = bluestore 2023-03-27T10:01:14.127 INFO:teuthology.orchestra.run.smithi023.stdout: osd op complaint time = 180 2023-03-27T10:01:14.127 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:01:14.127 INFO:teuthology.orchestra.run.smithi023.stdout:[mgr] 2023-03-27T10:01:14.127 INFO:teuthology.orchestra.run.smithi023.stdout: debug ms = 1 2023-03-27T10:01:14.127 INFO:teuthology.orchestra.run.smithi023.stdout: debug mgr = 20 2023-03-27T10:01:14.127 INFO:teuthology.orchestra.run.smithi023.stdout: debug mon = 20 2023-03-27T10:01:14.127 INFO:teuthology.orchestra.run.smithi023.stdout: debug auth = 20 2023-03-27T10:01:14.127 INFO:teuthology.orchestra.run.smithi023.stdout: mon reweight min pgs per osd = 4 2023-03-27T10:01:14.128 INFO:teuthology.orchestra.run.smithi023.stdout: mon reweight min bytes per osd = 10 2023-03-27T10:01:14.128 INFO:teuthology.orchestra.run.smithi023.stdout: mgr/telemetry/nag = false 2023-03-27T10:01:14.128 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:01:14.128 INFO:teuthology.orchestra.run.smithi023.stdout:[mon] 2023-03-27T10:01:14.128 INFO:teuthology.orchestra.run.smithi023.stdout: debug ms = 1 2023-03-27T10:01:14.128 INFO:teuthology.orchestra.run.smithi023.stdout: debug mon = 20 2023-03-27T10:01:14.128 INFO:teuthology.orchestra.run.smithi023.stdout: debug paxos = 20 2023-03-27T10:01:14.128 INFO:teuthology.orchestra.run.smithi023.stdout: debug auth = 20 2023-03-27T10:01:14.129 INFO:teuthology.orchestra.run.smithi023.stdout: mon data avail warn = 5 2023-03-27T10:01:14.129 INFO:teuthology.orchestra.run.smithi023.stdout: mon mgr mkfs grace = 240 2023-03-27T10:01:14.129 INFO:teuthology.orchestra.run.smithi023.stdout: mon reweight min pgs per osd = 4 2023-03-27T10:01:14.129 INFO:teuthology.orchestra.run.smithi023.stdout: mon osd reporter subtree level = osd 2023-03-27T10:01:14.129 INFO:teuthology.orchestra.run.smithi023.stdout: mon osd prime pg temp = true 2023-03-27T10:01:14.129 INFO:teuthology.orchestra.run.smithi023.stdout: mon reweight min bytes per osd = 10 2023-03-27T10:01:14.129 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:01:14.129 INFO:teuthology.orchestra.run.smithi023.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-03-27T10:01:14.130 INFO:teuthology.orchestra.run.smithi023.stdout: auth mon ticket ttl = 660 # 11m 2023-03-27T10:01:14.130 INFO:teuthology.orchestra.run.smithi023.stdout: auth service ticket ttl = 240 # 4m 2023-03-27T10:01:14.130 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:01:14.130 INFO:teuthology.orchestra.run.smithi023.stdout: # don't complain about insecure global_id in the test suite 2023-03-27T10:01:14.130 INFO:teuthology.orchestra.run.smithi023.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-03-27T10:01:14.130 INFO:teuthology.orchestra.run.smithi023.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-03-27T10:01:14.130 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:01:14.130 INFO:teuthology.orchestra.run.smithi023.stdout: # 1m isn't quite enough 2023-03-27T10:01:14.131 INFO:teuthology.orchestra.run.smithi023.stdout: mon_down_mkfs_grace = 2m 2023-03-27T10:01:14.131 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:01:14.131 INFO:teuthology.orchestra.run.smithi023.stdout: mon_warn_on_filestore_osds = false 2023-03-27T10:01:14.131 INFO:teuthology.orchestra.run.smithi023.stdout: mon op complaint time = 120 2023-03-27T10:01:14.131 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:01:14.131 INFO:teuthology.orchestra.run.smithi023.stdout:[client] 2023-03-27T10:01:14.131 INFO:teuthology.orchestra.run.smithi023.stdout: rgw cache enabled = true 2023-03-27T10:01:14.132 INFO:teuthology.orchestra.run.smithi023.stdout: rgw enable ops log = true 2023-03-27T10:01:14.132 INFO:teuthology.orchestra.run.smithi023.stdout: rgw enable usage log = true 2023-03-27T10:01:14.132 INFO:teuthology.orchestra.run.smithi023.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-03-27T10:01:14.132 INFO:teuthology.orchestra.run.smithi023.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-03-27T10:01:14.132 INFO:teuthology.orchestra.run.smithi023.stdout: client mount timeout = 600 2023-03-27T10:01:14.132 INFO:teuthology.orchestra.run.smithi023.stdout: debug client = 20 2023-03-27T10:01:14.132 INFO:teuthology.orchestra.run.smithi023.stdout: debug ms = 1 2023-03-27T10:01:14.132 INFO:teuthology.orchestra.run.smithi023.stdout: rados mon op timeout = 900 2023-03-27T10:01:14.133 INFO:teuthology.orchestra.run.smithi023.stdout: rados osd op timeout = 900 2023-03-27T10:01:14.133 INFO:teuthology.orchestra.run.smithi023.stdout:[mon.a] 2023-03-27T10:01:14.133 INFO:teuthology.orchestra.run.smithi023.stdout:[mon.b] 2023-03-27T10:01:14.133 INFO:teuthology.orchestra.run.smithi023.stdout:[mon.c] 2023-03-27T10:01:14.133 INFO:teuthology.orchestra.run.smithi023.stdout:[mds.a] 2023-03-27T10:01:14.133 INFO:teuthology.orchestra.run.smithi023.stdout:[mds.c] 2023-03-27T10:01:14.133 INFO:teuthology.orchestra.run.smithi023.stdout:[mds.e] 2023-03-27T10:01:14.133 INFO:teuthology.orchestra.run.smithi023.stdout:[mds.b] 2023-03-27T10:01:14.134 INFO:teuthology.orchestra.run.smithi023.stdout:[mds.d] 2023-03-27T10:01:14.134 INFO:teuthology.orchestra.run.smithi023.stdout:[mds.f] 2023-03-27T10:01:14.134 INFO:teuthology.orchestra.run.smithi023.stdout:[mds] 2023-03-27T10:01:14.134 INFO:teuthology.orchestra.run.smithi023.stdout: debug mds = 20 2023-03-27T10:01:14.134 INFO:teuthology.orchestra.run.smithi023.stdout: debug mds balancer = 20 2023-03-27T10:01:14.134 INFO:teuthology.orchestra.run.smithi023.stdout: debug ms = 1 2023-03-27T10:01:14.134 INFO:teuthology.orchestra.run.smithi023.stdout: mds bal fragment size max = 10000 2023-03-27T10:01:14.134 INFO:teuthology.orchestra.run.smithi023.stdout: mds bal merge size = 5 2023-03-27T10:01:14.135 INFO:teuthology.orchestra.run.smithi023.stdout: mds bal split bits = 3 2023-03-27T10:01:14.135 INFO:teuthology.orchestra.run.smithi023.stdout: mds bal split size = 100 2023-03-27T10:01:14.135 INFO:teuthology.orchestra.run.smithi023.stdout: mds debug frag = True 2023-03-27T10:01:14.135 INFO:teuthology.orchestra.run.smithi023.stdout: mds debug scatterstat = True 2023-03-27T10:01:14.135 INFO:teuthology.orchestra.run.smithi023.stdout: mds oft prefetch dirfrags = False 2023-03-27T10:01:14.135 INFO:teuthology.orchestra.run.smithi023.stdout: mds op complaint time = 180 2023-03-27T10:01:14.135 INFO:teuthology.orchestra.run.smithi023.stdout: mds verify scatter = True 2023-03-27T10:01:14.135 INFO:teuthology.orchestra.run.smithi023.stdout: mds_dir_prefetch = True 2023-03-27T10:01:14.136 INFO:teuthology.orchestra.run.smithi023.stdout: mds_sleep_rank_change = 5000000.0 2023-03-27T10:01:14.136 INFO:teuthology.orchestra.run.smithi023.stdout: osd op complaint time = 180 2023-03-27T10:01:14.136 INFO:teuthology.orchestra.run.smithi023.stdout: rados mon op timeout = 900 2023-03-27T10:01:14.136 INFO:teuthology.orchestra.run.smithi023.stdout: rados osd op timeout = 900 2023-03-27T10:01:14.140 INFO:tasks.ceph:Creating admin key on mon.a... 2023-03-27T10:01:14.141 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-03-27T10:01:14.243 INFO:tasks.ceph:Copying monmap to all nodes... 2023-03-27T10:01:14.243 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2023-03-27T10:01:14.244 DEBUG:teuthology.orchestra.run.smithi023:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-03-27T10:01:14.281 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2023-03-27T10:01:14.281 DEBUG:teuthology.orchestra.run.smithi023:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-03-27T10:01:14.359 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi023.front.sepia.ceph.com 2023-03-27T10:01:14.360 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2023-03-27T10:01:14.360 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd of=/etc/ceph/ceph.keyring 2023-03-27T10:01:14.360 DEBUG:teuthology.orchestra.run.smithi023:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-03-27T10:01:14.484 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2023-03-27T10:01:14.485 DEBUG:teuthology.orchestra.run.smithi023:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-03-27T10:01:14.564 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi112.front.sepia.ceph.com 2023-03-27T10:01:14.564 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-03-27T10:01:14.565 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.keyring 2023-03-27T10:01:14.565 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-03-27T10:01:14.603 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-03-27T10:01:14.604 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-03-27T10:01:14.665 INFO:tasks.ceph:Setting up mon nodes... 2023-03-27T10:01:14.666 INFO:tasks.ceph:Setting up mgr nodes... 2023-03-27T10:01:14.666 DEBUG:teuthology.orchestra.run.smithi023:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-03-27T10:01:14.779 INFO:teuthology.orchestra.run.smithi023.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-03-27T10:01:14.784 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-03-27T10:01:14.885 INFO:teuthology.orchestra.run.smithi112.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-03-27T10:01:14.888 INFO:tasks.ceph:Setting up mds nodes... 2023-03-27T10:01:14.888 DEBUG:teuthology.orchestra.run.smithi023:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-03-27T10:01:14.999 INFO:teuthology.orchestra.run.smithi023.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-03-27T10:01:15.004 DEBUG:teuthology.orchestra.run.smithi023:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-03-27T10:01:15.065 DEBUG:teuthology.orchestra.run.smithi023:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-03-27T10:01:15.177 INFO:teuthology.orchestra.run.smithi023.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-03-27T10:01:15.181 DEBUG:teuthology.orchestra.run.smithi023:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-03-27T10:01:15.242 DEBUG:teuthology.orchestra.run.smithi023:> sudo mkdir -p /var/lib/ceph/mds/ceph-e && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.e /var/lib/ceph/mds/ceph-e/keyring 2023-03-27T10:01:15.354 INFO:teuthology.orchestra.run.smithi023.stdout:creating /var/lib/ceph/mds/ceph-e/keyring 2023-03-27T10:01:15.359 DEBUG:teuthology.orchestra.run.smithi023:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-e 2023-03-27T10:01:15.421 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-03-27T10:01:15.481 INFO:teuthology.orchestra.run.smithi112.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-03-27T10:01:15.484 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-03-27T10:01:15.513 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-03-27T10:01:15.609 INFO:teuthology.orchestra.run.smithi112.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-03-27T10:01:15.611 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-03-27T10:01:15.638 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mds/ceph-f && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.f /var/lib/ceph/mds/ceph-f/keyring 2023-03-27T10:01:15.730 INFO:teuthology.orchestra.run.smithi112.stdout:creating /var/lib/ceph/mds/ceph-f/keyring 2023-03-27T10:01:15.733 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-f 2023-03-27T10:01:15.758 INFO:tasks.ceph_client:Setting up client nodes... 2023-03-27T10:01:15.759 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-03-27T10:01:15.848 INFO:teuthology.orchestra.run.smithi023.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-03-27T10:01:15.877 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-03-27T10:01:15.877 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi023.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi112.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-03-27T10:01:15.878 DEBUG:teuthology.orchestra.run.smithi023:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-03-27T10:01:15.938 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-03-27T10:01:15.939 INFO:tasks.ceph:role: osd.0 2023-03-27T10:01:15.939 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi023.front.sepia.ceph.com 2023-03-27T10:01:15.940 DEBUG:teuthology.orchestra.run.smithi023:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-03-27T10:01:16.027 INFO:teuthology.orchestra.run.smithi023.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-03-27T10:01:16.027 INFO:teuthology.orchestra.run.smithi023.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-27T10:01:16.028 INFO:teuthology.orchestra.run.smithi023.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-27T10:01:16.028 INFO:teuthology.orchestra.run.smithi023.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-27T10:01:16.028 INFO:teuthology.orchestra.run.smithi023.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-27T10:01:16.028 INFO:teuthology.orchestra.run.smithi023.stdout: = sunit=0 swidth=0 blks 2023-03-27T10:01:16.028 INFO:teuthology.orchestra.run.smithi023.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-27T10:01:16.028 INFO:teuthology.orchestra.run.smithi023.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-27T10:01:16.029 INFO:teuthology.orchestra.run.smithi023.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-27T10:01:16.029 INFO:teuthology.orchestra.run.smithi023.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-27T10:01:17.325 INFO:teuthology.orchestra.run.smithi023.stdout:Discarding blocks...Done. 2023-03-27T10:01:17.330 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi023.front.sepia.ceph.com -o noatime 2023-03-27T10:01:17.330 DEBUG:teuthology.orchestra.run.smithi023:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-03-27T10:01:17.437 DEBUG:teuthology.orchestra.run.smithi023:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-03-27T10:01:17.507 DEBUG:teuthology.orchestra.run.smithi023:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-03-27T10:01:17.570 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-03-27T10:01:17.571 INFO:tasks.ceph:role: osd.1 2023-03-27T10:01:17.571 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi023.front.sepia.ceph.com 2023-03-27T10:01:17.571 DEBUG:teuthology.orchestra.run.smithi023:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-03-27T10:01:17.663 INFO:teuthology.orchestra.run.smithi023.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-03-27T10:01:17.664 INFO:teuthology.orchestra.run.smithi023.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-27T10:01:17.664 INFO:teuthology.orchestra.run.smithi023.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-27T10:01:17.664 INFO:teuthology.orchestra.run.smithi023.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-27T10:01:17.664 INFO:teuthology.orchestra.run.smithi023.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-27T10:01:17.664 INFO:teuthology.orchestra.run.smithi023.stdout: = sunit=0 swidth=0 blks 2023-03-27T10:01:17.664 INFO:teuthology.orchestra.run.smithi023.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-27T10:01:17.665 INFO:teuthology.orchestra.run.smithi023.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-27T10:01:17.665 INFO:teuthology.orchestra.run.smithi023.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-27T10:01:17.665 INFO:teuthology.orchestra.run.smithi023.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-27T10:01:18.961 INFO:teuthology.orchestra.run.smithi023.stdout:Discarding blocks...Done. 2023-03-27T10:01:18.966 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi023.front.sepia.ceph.com -o noatime 2023-03-27T10:01:18.966 DEBUG:teuthology.orchestra.run.smithi023:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-03-27T10:01:19.075 DEBUG:teuthology.orchestra.run.smithi023:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-03-27T10:01:19.144 DEBUG:teuthology.orchestra.run.smithi023:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-03-27T10:01:19.205 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-03-27T10:01:19.205 INFO:tasks.ceph:role: osd.2 2023-03-27T10:01:19.205 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi023.front.sepia.ceph.com 2023-03-27T10:01:19.206 DEBUG:teuthology.orchestra.run.smithi023:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-03-27T10:01:19.294 INFO:teuthology.orchestra.run.smithi023.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-03-27T10:01:19.294 INFO:teuthology.orchestra.run.smithi023.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-27T10:01:19.294 INFO:teuthology.orchestra.run.smithi023.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-27T10:01:19.295 INFO:teuthology.orchestra.run.smithi023.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-27T10:01:19.295 INFO:teuthology.orchestra.run.smithi023.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-27T10:01:19.296 INFO:teuthology.orchestra.run.smithi023.stdout: = sunit=0 swidth=0 blks 2023-03-27T10:01:19.296 INFO:teuthology.orchestra.run.smithi023.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-27T10:01:19.296 INFO:teuthology.orchestra.run.smithi023.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-27T10:01:19.296 INFO:teuthology.orchestra.run.smithi023.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-27T10:01:19.297 INFO:teuthology.orchestra.run.smithi023.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-27T10:01:20.599 INFO:teuthology.orchestra.run.smithi023.stdout:Discarding blocks...Done. 2023-03-27T10:01:20.604 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi023.front.sepia.ceph.com -o noatime 2023-03-27T10:01:20.605 DEBUG:teuthology.orchestra.run.smithi023:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-03-27T10:01:20.714 DEBUG:teuthology.orchestra.run.smithi023:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-03-27T10:01:20.787 DEBUG:teuthology.orchestra.run.smithi023:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-03-27T10:01:20.848 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-03-27T10:01:20.849 INFO:tasks.ceph:role: osd.3 2023-03-27T10:01:20.849 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi023.front.sepia.ceph.com 2023-03-27T10:01:20.849 DEBUG:teuthology.orchestra.run.smithi023:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-03-27T10:01:20.937 INFO:teuthology.orchestra.run.smithi023.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-03-27T10:01:20.938 INFO:teuthology.orchestra.run.smithi023.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-27T10:01:20.938 INFO:teuthology.orchestra.run.smithi023.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-27T10:01:20.939 INFO:teuthology.orchestra.run.smithi023.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-27T10:01:20.939 INFO:teuthology.orchestra.run.smithi023.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-27T10:01:20.939 INFO:teuthology.orchestra.run.smithi023.stdout: = sunit=0 swidth=0 blks 2023-03-27T10:01:20.939 INFO:teuthology.orchestra.run.smithi023.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-27T10:01:20.940 INFO:teuthology.orchestra.run.smithi023.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-27T10:01:20.940 INFO:teuthology.orchestra.run.smithi023.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-27T10:01:20.940 INFO:teuthology.orchestra.run.smithi023.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-27T10:01:22.240 INFO:teuthology.orchestra.run.smithi023.stdout:Discarding blocks...Done. 2023-03-27T10:01:22.244 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi023.front.sepia.ceph.com -o noatime 2023-03-27T10:01:22.244 DEBUG:teuthology.orchestra.run.smithi023:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-03-27T10:01:22.350 DEBUG:teuthology.orchestra.run.smithi023:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-03-27T10:01:22.418 DEBUG:teuthology.orchestra.run.smithi023:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-27T10:01:22.530 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:22.530+0000 7f7a4dfdc580 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-03-27T10:01:22.531 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:22.531+0000 7f7a4dfdc580 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-03-27T10:01:22.531 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:22.531+0000 7f7a4dfdc580 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-03-27T10:01:22.531 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:22.531+0000 7f7a4dfdc580 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-03-27T10:01:22.531 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:22.531+0000 7f7a4dfdc580 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-03-27T10:01:22.532 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:22.534+0000 7f7a4dfdc580 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-03-27T10:01:25.426 DEBUG:teuthology.orchestra.run.smithi023:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-03-27T10:01:25.493 DEBUG:teuthology.orchestra.run.smithi023:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-27T10:01:25.592 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:25.593+0000 7f306bf13580 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-03-27T10:01:25.593 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:25.594+0000 7f306bf13580 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-03-27T10:01:25.593 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:25.594+0000 7f306bf13580 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-03-27T10:01:25.593 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:25.594+0000 7f306bf13580 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-03-27T10:01:25.593 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:25.594+0000 7f306bf13580 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-03-27T10:01:25.594 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:25.595+0000 7f306bf13580 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-03-27T10:01:28.493 DEBUG:teuthology.orchestra.run.smithi023:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-03-27T10:01:28.560 DEBUG:teuthology.orchestra.run.smithi023:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-27T10:01:28.661 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:28.661+0000 7f788890f580 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-03-27T10:01:28.662 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:28.662+0000 7f788890f580 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-03-27T10:01:28.662 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:28.662+0000 7f788890f580 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-03-27T10:01:28.662 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:28.662+0000 7f788890f580 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-03-27T10:01:28.662 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:28.662+0000 7f788890f580 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-03-27T10:01:28.663 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:28.663+0000 7f788890f580 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-03-27T10:01:31.573 DEBUG:teuthology.orchestra.run.smithi023:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-03-27T10:01:31.639 DEBUG:teuthology.orchestra.run.smithi023:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-27T10:01:31.736 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:31.737+0000 7fd8116ce580 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-03-27T10:01:31.737 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:31.738+0000 7fd8116ce580 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-03-27T10:01:31.737 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:31.738+0000 7fd8116ce580 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-03-27T10:01:31.737 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:31.738+0000 7fd8116ce580 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-03-27T10:01:31.737 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:31.738+0000 7fd8116ce580 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-03-27T10:01:31.738 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:31.740+0000 7fd8116ce580 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-03-27T10:01:34.649 DEBUG:teuthology.orchestra.run.smithi023:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-03-27T10:01:34.713 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-03-27T10:01:34.749 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-03-27T10:01:34.750 INFO:tasks.ceph:role: osd.4 2023-03-27T10:01:34.750 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi112.front.sepia.ceph.com 2023-03-27T10:01:34.751 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-03-27T10:01:34.826 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-03-27T10:01:34.826 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-27T10:01:34.827 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-27T10:01:34.827 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-27T10:01:34.827 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-27T10:01:34.827 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2023-03-27T10:01:34.827 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-27T10:01:34.827 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-27T10:01:34.827 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-27T10:01:34.828 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-27T10:01:35.258 INFO:teuthology.orchestra.run.smithi112.stdout:Discarding blocks...Done. 2023-03-27T10:01:35.260 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2023-03-27T10:01:35.261 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-03-27T10:01:35.418 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-03-27T10:01:35.454 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-03-27T10:01:35.522 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-03-27T10:01:35.523 INFO:tasks.ceph:role: osd.5 2023-03-27T10:01:35.523 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi112.front.sepia.ceph.com 2023-03-27T10:01:35.523 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-03-27T10:01:35.597 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-03-27T10:01:35.597 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-27T10:01:35.598 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-27T10:01:35.598 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-27T10:01:35.598 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-27T10:01:35.598 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2023-03-27T10:01:35.598 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-27T10:01:35.598 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-27T10:01:35.598 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-27T10:01:35.599 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-27T10:01:36.024 INFO:teuthology.orchestra.run.smithi112.stdout:Discarding blocks...Done. 2023-03-27T10:01:36.026 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2023-03-27T10:01:36.026 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-03-27T10:01:36.138 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-03-27T10:01:36.171 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-03-27T10:01:36.239 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-03-27T10:01:36.239 INFO:tasks.ceph:role: osd.6 2023-03-27T10:01:36.240 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi112.front.sepia.ceph.com 2023-03-27T10:01:36.240 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-03-27T10:01:36.314 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-03-27T10:01:36.314 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-27T10:01:36.314 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-27T10:01:36.314 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-27T10:01:36.314 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-27T10:01:36.315 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2023-03-27T10:01:36.315 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-27T10:01:36.315 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-27T10:01:36.315 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-27T10:01:36.315 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-27T10:01:36.772 INFO:teuthology.orchestra.run.smithi112.stdout:Discarding blocks...Done. 2023-03-27T10:01:36.774 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2023-03-27T10:01:36.774 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-03-27T10:01:36.933 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-03-27T10:01:36.967 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-03-27T10:01:37.037 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-03-27T10:01:37.037 INFO:tasks.ceph:role: osd.7 2023-03-27T10:01:37.038 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi112.front.sepia.ceph.com 2023-03-27T10:01:37.038 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-03-27T10:01:37.112 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-03-27T10:01:37.112 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-27T10:01:37.112 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-27T10:01:37.112 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-27T10:01:37.113 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-27T10:01:37.113 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2023-03-27T10:01:37.113 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-27T10:01:37.113 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-27T10:01:37.114 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-27T10:01:37.114 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-27T10:01:37.544 INFO:teuthology.orchestra.run.smithi112.stdout:Discarding blocks...Done. 2023-03-27T10:01:37.546 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2023-03-27T10:01:37.546 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-03-27T10:01:37.710 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-03-27T10:01:37.747 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-27T10:01:37.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:37.863+0000 7ff08cab1580 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-03-27T10:01:37.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:37.863+0000 7ff08cab1580 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-03-27T10:01:37.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:37.863+0000 7ff08cab1580 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-03-27T10:01:37.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:37.863+0000 7ff08cab1580 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-03-27T10:01:37.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:37.863+0000 7ff08cab1580 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-03-27T10:01:37.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:37.863+0000 7ff08cab1580 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-03-27T10:01:40.730 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-03-27T10:01:40.761 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-27T10:01:40.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:40.852+0000 7fe6cd49f580 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-03-27T10:01:40.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:40.852+0000 7fe6cd49f580 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-03-27T10:01:40.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:40.852+0000 7fe6cd49f580 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-03-27T10:01:40.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:40.852+0000 7fe6cd49f580 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-03-27T10:01:40.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:40.852+0000 7fe6cd49f580 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-03-27T10:01:40.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:40.853+0000 7fe6cd49f580 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-03-27T10:01:41.688 DEBUG:teuthology.orchestra.run.smithi023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-27T10:01:41.692 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-27T10:01:43.722 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-03-27T10:01:43.754 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-27T10:01:43.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:43.847+0000 7f19a911c580 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-03-27T10:01:43.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:43.847+0000 7f19a911c580 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-03-27T10:01:43.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:43.847+0000 7f19a911c580 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-03-27T10:01:43.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:43.847+0000 7f19a911c580 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-03-27T10:01:43.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:43.847+0000 7f19a911c580 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-03-27T10:01:43.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:43.848+0000 7f19a911c580 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-03-27T10:01:46.706 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-03-27T10:01:46.737 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-27T10:01:46.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:46.826+0000 7ffbbf4f1580 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-03-27T10:01:46.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:46.827+0000 7ffbbf4f1580 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-03-27T10:01:46.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:46.827+0000 7ffbbf4f1580 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-03-27T10:01:46.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:46.827+0000 7ffbbf4f1580 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-03-27T10:01:46.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:46.827+0000 7ffbbf4f1580 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-03-27T10:01:46.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:46.827+0000 7ffbbf4f1580 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-03-27T10:01:49.690 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-03-27T10:01:49.722 INFO:tasks.ceph:Reading keys from all nodes... 2023-03-27T10:01:49.723 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2023-03-27T10:01:49.723 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-03-27T10:01:49.790 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2023-03-27T10:01:49.791 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-03-27T10:01:49.893 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2023-03-27T10:01:49.894 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-03-27T10:01:49.994 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2023-03-27T10:01:49.995 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd if=/var/lib/ceph/mds/ceph-e/keyring of=/dev/stdout 2023-03-27T10:01:50.095 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2023-03-27T10:01:50.095 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-03-27T10:01:50.195 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2023-03-27T10:01:50.195 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-03-27T10:01:50.295 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2023-03-27T10:01:50.295 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-03-27T10:01:50.395 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2023-03-27T10:01:50.396 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-03-27T10:01:50.495 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-03-27T10:01:50.496 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-03-27T10:01:50.528 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-03-27T10:01:50.529 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-03-27T10:01:50.598 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-03-27T10:01:50.599 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-03-27T10:01:50.669 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-03-27T10:01:50.670 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/mds/ceph-f/keyring of=/dev/stdout 2023-03-27T10:01:50.738 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-03-27T10:01:50.739 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-03-27T10:01:50.807 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-03-27T10:01:50.807 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-03-27T10:01:50.874 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-03-27T10:01:50.875 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-03-27T10:01:50.943 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-03-27T10:01:50.944 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-03-27T10:01:51.011 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2023-03-27T10:01:51.012 DEBUG:teuthology.orchestra.run.smithi023:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-03-27T10:01:51.051 INFO:tasks.ceph:Adding keys to all mons... 2023-03-27T10:01:51.052 DEBUG:teuthology.orchestra.run.smithi023:> sudo tee -a /etc/ceph/ceph.keyring 2023-03-27T10:01:51.095 DEBUG:teuthology.orchestra.run.smithi112:> sudo tee -a /etc/ceph/ceph.keyring 2023-03-27T10:01:51.121 INFO:teuthology.orchestra.run.smithi112.stdout:[mgr.y] 2023-03-27T10:01:51.122 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQBqaSFkNh2NLhAAzeIUJXyb2u/SpUXGcVSh+w== 2023-03-27T10:01:51.122 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.a] 2023-03-27T10:01:51.122 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQBraSFkitcYABAAPn24WYUYnxGy9afXsV95nA== 2023-03-27T10:01:51.122 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.c] 2023-03-27T10:01:51.122 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQBraSFk7RCsChAAbtXBr0/IJ4sXOu1/UHeqHQ== 2023-03-27T10:01:51.122 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.e] 2023-03-27T10:01:51.123 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQBraSFkdOZAFRAAuz/EanRrOon04Y88qMnXlA== 2023-03-27T10:01:51.123 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.0] 2023-03-27T10:01:51.123 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQByaSFkM1mpHxAAXmHyporwzO4fotzqmV3e1Q== 2023-03-27T10:01:51.123 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.1] 2023-03-27T10:01:51.123 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQB1aSFkGOloIxAA4kmcZkrxaGLG3TzPxPOPog== 2023-03-27T10:01:51.124 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.2] 2023-03-27T10:01:51.124 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQB4aSFkEd12JxAATwQ1Avh6UAoOoDJFlUUGDw== 2023-03-27T10:01:51.124 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.3] 2023-03-27T10:01:51.125 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQB7aSFkcxUBLBAAPapBhVnl/6pWA1/Fz7ixlA== 2023-03-27T10:01:51.125 INFO:teuthology.orchestra.run.smithi112.stdout:[mgr.x] 2023-03-27T10:01:51.125 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQBqaSFkZYDlNBAApL2ub/HBGoiEXUJ1zi8Zwg== 2023-03-27T10:01:51.125 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.b] 2023-03-27T10:01:51.125 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQBraSFkSjvWHBAAYet/UxCyAvX1DX8K4dANMw== 2023-03-27T10:01:51.125 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.d] 2023-03-27T10:01:51.126 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQBraSFkjqVqJBAAPRXJCjwOONbOFIF/38UL1w== 2023-03-27T10:01:51.126 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.f] 2023-03-27T10:01:51.126 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQBraSFknNmuKxAANjQ6l4oqfWFw853LVL/qNw== 2023-03-27T10:01:51.126 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.4] 2023-03-27T10:01:51.126 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQCBaSFkNAl8MxAAjcxQZqmgaYKyff+2XcxV/A== 2023-03-27T10:01:51.126 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.5] 2023-03-27T10:01:51.127 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQCEaSFkZlHaMhAAmOHq2HQteEXZwp0MKKgpiA== 2023-03-27T10:01:51.127 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.6] 2023-03-27T10:01:51.127 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQCHaSFkw5KOMhAAyu+2lOd5WlAaIlgbFwSQvg== 2023-03-27T10:01:51.127 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.7] 2023-03-27T10:01:51.127 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQCKaSFkLdxhMRAAXDq7z970PJN5ItdXR8HMXw== 2023-03-27T10:01:51.127 INFO:teuthology.orchestra.run.smithi112.stdout:[client.0] 2023-03-27T10:01:51.128 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQBraSFkIGexMhAAVDNISVtJoqBBk6X1zcNpdg== 2023-03-27T10:01:51.150 INFO:teuthology.orchestra.run.smithi023.stdout:[mgr.y] 2023-03-27T10:01:51.150 INFO:teuthology.orchestra.run.smithi023.stdout: key = AQBqaSFkNh2NLhAAzeIUJXyb2u/SpUXGcVSh+w== 2023-03-27T10:01:51.150 INFO:teuthology.orchestra.run.smithi023.stdout:[mds.a] 2023-03-27T10:01:51.151 INFO:teuthology.orchestra.run.smithi023.stdout: key = AQBraSFkitcYABAAPn24WYUYnxGy9afXsV95nA== 2023-03-27T10:01:51.151 INFO:teuthology.orchestra.run.smithi023.stdout:[mds.c] 2023-03-27T10:01:51.151 INFO:teuthology.orchestra.run.smithi023.stdout: key = AQBraSFk7RCsChAAbtXBr0/IJ4sXOu1/UHeqHQ== 2023-03-27T10:01:51.151 INFO:teuthology.orchestra.run.smithi023.stdout:[mds.e] 2023-03-27T10:01:51.152 INFO:teuthology.orchestra.run.smithi023.stdout: key = AQBraSFkdOZAFRAAuz/EanRrOon04Y88qMnXlA== 2023-03-27T10:01:51.152 INFO:teuthology.orchestra.run.smithi023.stdout:[osd.0] 2023-03-27T10:01:51.152 INFO:teuthology.orchestra.run.smithi023.stdout: key = AQByaSFkM1mpHxAAXmHyporwzO4fotzqmV3e1Q== 2023-03-27T10:01:51.152 INFO:teuthology.orchestra.run.smithi023.stdout:[osd.1] 2023-03-27T10:01:51.153 INFO:teuthology.orchestra.run.smithi023.stdout: key = AQB1aSFkGOloIxAA4kmcZkrxaGLG3TzPxPOPog== 2023-03-27T10:01:51.153 INFO:teuthology.orchestra.run.smithi023.stdout:[osd.2] 2023-03-27T10:01:51.153 INFO:teuthology.orchestra.run.smithi023.stdout: key = AQB4aSFkEd12JxAATwQ1Avh6UAoOoDJFlUUGDw== 2023-03-27T10:01:51.153 INFO:teuthology.orchestra.run.smithi023.stdout:[osd.3] 2023-03-27T10:01:51.154 INFO:teuthology.orchestra.run.smithi023.stdout: key = AQB7aSFkcxUBLBAAPapBhVnl/6pWA1/Fz7ixlA== 2023-03-27T10:01:51.154 INFO:teuthology.orchestra.run.smithi023.stdout:[mgr.x] 2023-03-27T10:01:51.154 INFO:teuthology.orchestra.run.smithi023.stdout: key = AQBqaSFkZYDlNBAApL2ub/HBGoiEXUJ1zi8Zwg== 2023-03-27T10:01:51.154 INFO:teuthology.orchestra.run.smithi023.stdout:[mds.b] 2023-03-27T10:01:51.155 INFO:teuthology.orchestra.run.smithi023.stdout: key = AQBraSFkSjvWHBAAYet/UxCyAvX1DX8K4dANMw== 2023-03-27T10:01:51.155 INFO:teuthology.orchestra.run.smithi023.stdout:[mds.d] 2023-03-27T10:01:51.155 INFO:teuthology.orchestra.run.smithi023.stdout: key = AQBraSFkjqVqJBAAPRXJCjwOONbOFIF/38UL1w== 2023-03-27T10:01:51.155 INFO:teuthology.orchestra.run.smithi023.stdout:[mds.f] 2023-03-27T10:01:51.156 INFO:teuthology.orchestra.run.smithi023.stdout: key = AQBraSFknNmuKxAANjQ6l4oqfWFw853LVL/qNw== 2023-03-27T10:01:51.156 INFO:teuthology.orchestra.run.smithi023.stdout:[osd.4] 2023-03-27T10:01:51.156 INFO:teuthology.orchestra.run.smithi023.stdout: key = AQCBaSFkNAl8MxAAjcxQZqmgaYKyff+2XcxV/A== 2023-03-27T10:01:51.156 INFO:teuthology.orchestra.run.smithi023.stdout:[osd.5] 2023-03-27T10:01:51.157 INFO:teuthology.orchestra.run.smithi023.stdout: key = AQCEaSFkZlHaMhAAmOHq2HQteEXZwp0MKKgpiA== 2023-03-27T10:01:51.157 INFO:teuthology.orchestra.run.smithi023.stdout:[osd.6] 2023-03-27T10:01:51.157 INFO:teuthology.orchestra.run.smithi023.stdout: key = AQCHaSFkw5KOMhAAyu+2lOd5WlAaIlgbFwSQvg== 2023-03-27T10:01:51.157 INFO:teuthology.orchestra.run.smithi023.stdout:[osd.7] 2023-03-27T10:01:51.158 INFO:teuthology.orchestra.run.smithi023.stdout: key = AQCKaSFkLdxhMRAAXDq7z970PJN5ItdXR8HMXw== 2023-03-27T10:01:51.158 INFO:teuthology.orchestra.run.smithi023.stdout:[client.0] 2023-03-27T10:01:51.158 INFO:teuthology.orchestra.run.smithi023.stdout: key = AQBraSFkIGexMhAAVDNISVtJoqBBk6X1zcNpdg== 2023-03-27T10:01:51.160 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-03-27T10:01:51.205 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-03-27T10:01:51.299 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-27T10:01:51.304 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-27T10:01:51.402 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-27T10:01:51.407 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-27T10:01:51.505 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-27T10:01:51.509 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-27T10:01:51.608 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-27T10:01:51.614 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-27T10:01:51.709 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-27T10:01:51.713 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-27T10:01:51.809 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-27T10:01:51.813 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-27T10:01:51.911 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-27T10:01:51.915 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-27T10:01:52.013 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-03-27T10:01:52.017 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-03-27T10:01:52.113 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-27T10:01:52.118 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-27T10:01:52.217 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-27T10:01:52.221 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-27T10:01:52.321 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-27T10:01:52.326 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-27T10:01:52.430 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-27T10:01:52.434 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-27T10:01:52.529 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-27T10:01:52.534 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-27T10:01:52.629 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-27T10:01:52.636 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-27T10:01:52.732 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-27T10:01:52.737 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-27T10:01:52.834 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-27T10:01:52.839 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-27T10:01:52.938 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-03-27T10:01:52.939 DEBUG:teuthology.orchestra.run.smithi023:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-03-27T10:01:53.000 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-03-27T10:01:53.115 DEBUG:teuthology.orchestra.run.smithi023:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-03-27T10:01:53.175 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-03-27T10:01:53.203 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-03-27T10:01:53.300 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-03-27T10:01:53.326 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-03-27T10:01:53.393 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-03-27T10:01:53.491 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-03-27T10:01:53.517 DEBUG:teuthology.orchestra.run.smithi023:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-03-27T10:01:53.524 DEBUG:teuthology.orchestra.run.smithi112:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-03-27T10:01:53.574 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-03-27T10:01:53.575 INFO:tasks.ceph.mon.a:Restarting daemon 2023-03-27T10:01:53.575 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-03-27T10:01:53.603 INFO:tasks.ceph.mon.a:Started 2023-03-27T10:01:53.604 INFO:tasks.ceph.mon.b:Restarting daemon 2023-03-27T10:01:53.604 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-03-27T10:01:53.617 INFO:tasks.ceph.mon.b:Started 2023-03-27T10:01:53.617 INFO:tasks.ceph.mon.c:Restarting daemon 2023-03-27T10:01:53.618 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-03-27T10:01:53.620 INFO:tasks.ceph.mon.c:Started 2023-03-27T10:01:53.620 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-03-27T10:01:53.620 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-03-27T10:01:53.621 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-03-27T10:01:53.625 INFO:tasks.ceph.mgr.y:Started 2023-03-27T10:01:53.626 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-03-27T10:01:53.626 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-03-27T10:01:53.628 INFO:tasks.ceph.mgr.x:Started 2023-03-27T10:01:53.628 INFO:tasks.ceph:Setting crush tunables to default 2023-03-27T10:01:53.628 DEBUG:teuthology.orchestra.run.smithi023:> sudo ceph --cluster ceph osd crush tunables default 2023-03-27T10:01:53.818 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.819+0000 7fc75dfff700 1 Processor -- start 2023-03-27T10:01:53.819 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.819+0000 7fc75dfff700 1 -- start start 2023-03-27T10:01:53.820 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.820+0000 7fc75dfff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7580c6990 0x7fc7580c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:53.821 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.820+0000 7fc75dfff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc7580c9520 0x7fc7580c8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:53.821 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.820+0000 7fc75dfff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7580ca730 con 0x7fc75806b440 2023-03-27T10:01:53.821 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.820+0000 7fc75dfff700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7580c72a0 con 0x7fc7580c9520 2023-03-27T10:01:53.821 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.820+0000 7fc75dfff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7580cf9f0 con 0x7fc7580c6990 2023-03-27T10:01:53.821 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.820+0000 7fc756ffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc7580c9520 0x7fc7580c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:53.822 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.821+0000 7fc7577fe700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7580c6990 0x7fc7580c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:53.822 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.821+0000 7fc756ffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc7580c9520 0x7fc7580c8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.23:36258/0 (socket says 172.21.15.23:36258) 2023-03-27T10:01:53.822 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.821+0000 7fc757fff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fc75806b440 0x7fc75806b860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.23:58872/0 (socket says 172.21.15.23:58872) 2023-03-27T10:01:53.822 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.821+0000 7fc756ffd700 1 -- 172.21.15.23:0/506443035 learned_addr learned my addr 172.21.15.23:0/506443035 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:01:53.822 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.821+0000 7fc756ffd700 1 -- 172.21.15.23:0/506443035 >> v2:172.21.15.112:3300/0 conn(0x7fc7580c9520 msgr2=0x7fc7580c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-03-27T10:01:53.822 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.821+0000 7fc756ffd700 1 -- 172.21.15.23:0/506443035 >> v2:172.21.15.112:3300/0 conn(0x7fc7580c9520 msgr2=0x7fc7580c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-03-27T10:01:53.822 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.821+0000 7fc756ffd700 1 --2- 172.21.15.23:0/506443035 >> v2:172.21.15.112:3300/0 conn(0x7fc7580c9520 0x7fc7580c8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-03-27T10:01:53.823 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.821+0000 7fc7577fe700 1 -- 172.21.15.23:0/506443035 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7580c6990 msgr2=0x7fc7580c6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-03-27T10:01:53.823 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.821+0000 7fc7577fe700 1 -- 172.21.15.23:0/506443035 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7580c6990 msgr2=0x7fc7580c6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-03-27T10:01:53.823 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.821+0000 7fc7577fe700 1 --2- 172.21.15.23:0/506443035 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7580c6990 0x7fc7580c6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-03-27T10:01:53.823 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.821+0000 7fc756ffd700 1 --2- 172.21.15.23:0/506443035 >> v2:172.21.15.112:3300/0 conn(0x7fc7580c9520 0x7fc7580c8820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-03-27T10:01:53.823 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.821+0000 7fc7577fe700 1 --2- 172.21.15.23:0/506443035 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7580c6990 0x7fc7580c6d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-03-27T10:01:53.839 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.840+0000 7fc7567fc700 1 -- 172.21.15.23:0/506443035 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 56837006 0 0) 0x7fc7580ca730 con 0x7fc75806b440 2023-03-27T10:01:53.840 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.840+0000 7fc7567fc700 1 -- 172.21.15.23:0/506443035 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7400035b0 con 0x7fc75806b440 2023-03-27T10:01:53.845 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.846+0000 7fc7567fc700 1 -- 172.21.15.23:0/506443035 <== mon.1 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3266787561 0 0) 0x7fc7400035b0 con 0x7fc75806b440 2023-03-27T10:01:53.846 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.846+0000 7fc7567fc700 1 -- 172.21.15.23:0/506443035 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7580c6990 msgr2=0x7fc7580c6d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-03-27T10:01:53.846 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.846+0000 7fc7567fc700 1 --2- 172.21.15.23:0/506443035 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7580c6990 0x7fc7580c6d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:53.846 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.846+0000 7fc7567fc700 1 -- 172.21.15.23:0/506443035 >> v2:172.21.15.112:3300/0 conn(0x7fc7580c9520 msgr2=0x7fc7580c8820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-03-27T10:01:53.846 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.846+0000 7fc7567fc700 1 --2- 172.21.15.23:0/506443035 >> v2:172.21.15.112:3300/0 conn(0x7fc7580c9520 0x7fc7580c8820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:53.848 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.847+0000 7fc7567fc700 1 -- 172.21.15.23:0/506443035 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc758158a00 con 0x7fc75806b440 2023-03-27T10:01:53.848 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.847+0000 7fc7567fc700 1 -- 172.21.15.23:0/506443035 <== mon.1 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3881486710 0 0) 0x7fc7500019f0 con 0x7fc75806b440 2023-03-27T10:01:53.848 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.847+0000 7fc7567fc700 1 -- 172.21.15.23:0/506443035 >> v1:172.21.15.112:6789/0 conn(0x7fc75806b440 legacy=0x7fc75806b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:53.848 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.847+0000 7fc7567fc700 1 --2- 172.21.15.23:0/506443035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc740003a00 0x7fc740003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:53.848 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.847+0000 7fc7567fc700 1 --2- 172.21.15.23:0/506443035 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7580c6990 0x7fc740004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:53.849 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.847+0000 7fc7567fc700 1 --2- 172.21.15.23:0/506443035 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fc740004d90 0x7fc740005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:53.849 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.847+0000 7fc7567fc700 1 -- 172.21.15.23:0/506443035 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7580c72a0 con 0x7fc740004d90 2023-03-27T10:01:53.849 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.847+0000 7fc7567fc700 1 -- 172.21.15.23:0/506443035 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7580cf9f0 con 0x7fc740003a00 2023-03-27T10:01:53.849 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.847+0000 7fc7567fc700 1 -- 172.21.15.23:0/506443035 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc74000baf0 con 0x7fc7580c6990 2023-03-27T10:01:53.849 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.847+0000 7fc7577fe700 1 --2- 172.21.15.23:0/506443035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc740003a00 0x7fc740003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:53.849 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.847+0000 7fc756ffd700 1 --2- 172.21.15.23:0/506443035 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7580c6990 0x7fc740004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:53.849 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.847+0000 7fc757fff700 1 --2- 172.21.15.23:0/506443035 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fc740004d90 0x7fc740005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:53.850 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.847+0000 7fc7577fe700 1 -- 172.21.15.23:0/506443035 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7580c6990 msgr2=0x7fc740004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:01:53.850 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.847+0000 7fc7577fe700 1 --2- 172.21.15.23:0/506443035 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7580c6990 0x7fc740004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:53.850 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.847+0000 7fc7577fe700 1 -- 172.21.15.23:0/506443035 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fc740004d90 msgr2=0x7fc740005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:01:53.850 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.847+0000 7fc7577fe700 1 --2- 172.21.15.23:0/506443035 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fc740004d90 0x7fc740005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:53.850 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.847+0000 7fc7577fe700 1 -- 172.21.15.23:0/506443035 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7400035b0 con 0x7fc740003a00 2023-03-27T10:01:53.850 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.847+0000 7fc757fff700 1 --2- 172.21.15.23:0/506443035 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fc740004d90 0x7fc740005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-27T10:01:53.850 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.848+0000 7fc7577fe700 1 --2- 172.21.15.23:0/506443035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc740003a00 0x7fc740003e10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc748009890 tx=0x7fc740005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45744cef16fdd7ed server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:01:53.851 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.848+0000 7fc7567fc700 1 -- 172.21.15.23:0/506443035 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74800c070 con 0x7fc740003a00 2023-03-27T10:01:53.851 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.848+0000 7fc7567fc700 1 -- 172.21.15.23:0/506443035 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc748004020 con 0x7fc740003a00 2023-03-27T10:01:53.851 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.848+0000 7fc7567fc700 1 -- 172.21.15.23:0/506443035 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7480043d0 con 0x7fc740003a00 2023-03-27T10:01:53.853 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.849+0000 7fc75dfff700 1 -- 172.21.15.23:0/506443035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc740003a00 msgr2=0x7fc740003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:53.853 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.849+0000 7fc75dfff700 1 --2- 172.21.15.23:0/506443035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc740003a00 0x7fc740003e10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc748009890 tx=0x7fc740005880 comp rx=0 tx=0).stop 2023-03-27T10:01:53.853 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.849+0000 7fc757fff700 1 -- 172.21.15.23:0/506443035 reap_dead start 2023-03-27T10:01:53.853 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.849+0000 7fc75dfff700 1 -- 172.21.15.23:0/506443035 shutdown_connections 2023-03-27T10:01:53.853 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.849+0000 7fc75dfff700 1 -- 172.21.15.23:0/506443035 >> 172.21.15.23:0/506443035 conn(0x7fc758071920 msgr2=0x7fc758071cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:01:53.854 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.849+0000 7fc75dfff700 1 -- 172.21.15.23:0/506443035 shutdown_connections 2023-03-27T10:01:53.854 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.849+0000 7fc75dfff700 1 -- 172.21.15.23:0/506443035 wait complete. 2023-03-27T10:01:53.854 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.851+0000 7fc75dfff700 1 Processor -- start 2023-03-27T10:01:53.854 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.851+0000 7fc75dfff700 1 -- start start 2023-03-27T10:01:53.854 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.852+0000 7fc75dfff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fc740003a00 0x7fc758065f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:53.854 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.852+0000 7fc75dfff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7581599c0 0x7fc7581ef6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:53.854 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.852+0000 7fc75dfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc758150440 0x7fc7581efc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:53.855 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.852+0000 7fc75dfff700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7581f23f0 con 0x7fc740003a00 2023-03-27T10:01:53.855 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.852+0000 7fc75dfff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7581f15b0 con 0x7fc758150440 2023-03-27T10:01:53.855 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.852+0000 7fc75dfff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc758150b70 con 0x7fc7581599c0 2023-03-27T10:01:53.855 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.852+0000 7fc756ffd700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7581599c0 0x7fc7581ef6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:53.855 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.852+0000 7fc756ffd700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7581599c0 0x7fc7581ef6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.23:39864/0 (socket says 172.21.15.23:39864) 2023-03-27T10:01:53.855 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.852+0000 7fc756ffd700 1 -- 172.21.15.23:0/2868849515 learned_addr learned my addr 172.21.15.23:0/2868849515 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:01:53.855 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.853+0000 7fc757fff700 1 --2- 172.21.15.23:0/2868849515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc758150440 0x7fc7581efc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:53.855 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.853+0000 7fc7577fe700 1 --2- 172.21.15.23:0/2868849515 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fc740003a00 0x7fc758065f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:53.856 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.853+0000 7fc756ffd700 1 -- 172.21.15.23:0/2868849515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc758150440 msgr2=0x7fc7581efc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:53.856 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.853+0000 7fc756ffd700 1 --2- 172.21.15.23:0/2868849515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc758150440 0x7fc7581efc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:53.856 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.853+0000 7fc756ffd700 1 -- 172.21.15.23:0/2868849515 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fc740003a00 msgr2=0x7fc758065f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:53.856 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.853+0000 7fc756ffd700 1 --2- 172.21.15.23:0/2868849515 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fc740003a00 0x7fc758065f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:53.856 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.853+0000 7fc756ffd700 1 -- 172.21.15.23:0/2868849515 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc758158fc0 con 0x7fc7581599c0 2023-03-27T10:01:53.856 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.853+0000 7fc757fff700 1 --2- 172.21.15.23:0/2868849515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc758150440 0x7fc7581efc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-27T10:01:53.856 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.853+0000 7fc756ffd700 1 --2- 172.21.15.23:0/2868849515 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7581599c0 0x7fc7581ef6f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc74c011900 tx=0x7fc74c011c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:01:53.857 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.854+0000 7fc73ffff700 1 -- 172.21.15.23:0/2868849515 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c00c070 con 0x7fc7581599c0 2023-03-27T10:01:53.857 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.854+0000 7fc73ffff700 1 -- 172.21.15.23:0/2868849515 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74c00f8d0 con 0x7fc7581599c0 2023-03-27T10:01:53.857 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.854+0000 7fc75dfff700 1 -- 172.21.15.23:0/2868849515 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc758158ce0 con 0x7fc7581599c0 2023-03-27T10:01:53.857 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.854+0000 7fc73ffff700 1 -- 172.21.15.23:0/2868849515 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c009620 con 0x7fc7581599c0 2023-03-27T10:01:53.858 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.854+0000 7fc75dfff700 1 -- 172.21.15.23:0/2868849515 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7581592a0 con 0x7fc7581599c0 2023-03-27T10:01:53.858 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.854+0000 7fc73ffff700 1 -- 172.21.15.23:0/2868849515 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fc74c025070 con 0x7fc7581599c0 2023-03-27T10:01:53.858 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.854+0000 7fc73ffff700 1 -- 172.21.15.23:0/2868849515 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc74c015cd0 con 0x7fc7581599c0 2023-03-27T10:01:53.859 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.855+0000 7fc75dfff700 1 -- 172.21.15.23:0/2868849515 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7581ef410 con 0x7fc7581599c0 2023-03-27T10:01:53.859 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.858+0000 7fc73ffff700 1 -- 172.21.15.23:0/2868849515 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72400 (secure 0 0 0) 0x7fc7581ef410 con 0x7fc7581599c0 2023-03-27T10:01:53.937 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.938+0000 7fc75dfff700 1 -- 172.21.15.23:0/2868849515 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fc758069220 con 0x7fc7581599c0 2023-03-27T10:01:53.940 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.941+0000 7fc73ffff700 1 -- 172.21.15.23:0/2868849515 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fc74c02d090 con 0x7fc7581599c0 2023-03-27T10:01:53.942 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.943+0000 7fc75dfff700 1 -- 172.21.15.23:0/2868849515 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7581599c0 msgr2=0x7fc7581ef6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:53.943 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.943+0000 7fc75dfff700 1 --2- 172.21.15.23:0/2868849515 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7581599c0 0x7fc7581ef6f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc74c011900 tx=0x7fc74c011c10 comp rx=0 tx=0).stop 2023-03-27T10:01:53.943 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.943+0000 7fc75dfff700 1 -- 172.21.15.23:0/2868849515 shutdown_connections 2023-03-27T10:01:53.943 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.943+0000 7fc75dfff700 1 --2- 172.21.15.23:0/2868849515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc758150440 0x7fc7581efc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:53.943 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.943+0000 7fc75dfff700 1 --2- 172.21.15.23:0/2868849515 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fc740003a00 0x7fc758065f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:53.943 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.943+0000 7fc75dfff700 1 --2- 172.21.15.23:0/2868849515 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7581599c0 0x7fc7581ef6f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:53.943 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.943+0000 7fc75dfff700 1 -- 172.21.15.23:0/2868849515 >> 172.21.15.23:0/2868849515 conn(0x7fc758071920 msgr2=0x7fc7580cda50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:01:53.944 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.943+0000 7fc75dfff700 1 -- 172.21.15.23:0/2868849515 shutdown_connections 2023-03-27T10:01:53.944 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:53.943+0000 7fc75dfff700 1 -- 172.21.15.23:0/2868849515 wait complete. 2023-03-27T10:01:53.944 INFO:teuthology.orchestra.run.smithi023.stderr:adjusted tunables profile to default 2023-03-27T10:01:53.959 INFO:tasks.ceph:check_enable_crimson: False 2023-03-27T10:01:53.960 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-03-27T10:01:53.961 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2023-03-27T10:01:53.961 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-03-27T10:01:53.962 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-03-27T10:01:53.962+0000 7f105b615200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-03-27T10:01:53.994 INFO:tasks.ceph.mgr.y.smithi023.stderr:2023-03-27T10:01:53.991+0000 7f6256375200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-03-27T10:01:54.023 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2023-03-27T10:01:54.024 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-03-27T10:01:54.026 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-03-27T10:01:54.025+0000 7f105b615200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-03-27T10:01:54.073 INFO:tasks.ceph.mgr.y.smithi023.stderr:2023-03-27T10:01:54.073+0000 7f6256375200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-03-27T10:01:54.098 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-03-27T10:01:54.099+0000 7f105b615200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-03-27T10:01:54.125 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2023-03-27T10:01:54.125 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-03-27T10:01:54.170 INFO:tasks.ceph.mgr.y.smithi023.stderr:2023-03-27T10:01:54.169+0000 7f6256375200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-03-27T10:01:54.175 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-03-27T10:01:54.175+0000 7f105b615200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-03-27T10:01:54.227 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2023-03-27T10:01:54.227 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-03-27T10:01:54.272 INFO:tasks.ceph.mgr.y.smithi023.stderr:2023-03-27T10:01:54.270+0000 7f6256375200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-03-27T10:01:54.312 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-03-27T10:01:54.312+0000 7f105b615200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-03-27T10:01:54.329 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-03-27T10:01:54.329 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-03-27T10:01:54.356 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-03-27T10:01:54.356 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-03-27T10:01:54.422 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-03-27T10:01:54.422 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-03-27T10:01:54.449 INFO:tasks.ceph.mgr.y.smithi023.stderr:2023-03-27T10:01:54.449+0000 7f6256375200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-03-27T10:01:54.488 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-03-27T10:01:54.488 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-03-27T10:01:54.555 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new df086052-536b-4fb1-bcce-7aaf5a13911e 0 2023-03-27T10:01:54.668 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-03-27T10:01:54.668+0000 7f105b615200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-03-27T10:01:54.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.715+0000 7ff71f9db700 1 Processor -- start 2023-03-27T10:01:54.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.716+0000 7ff71f9db700 1 -- start start 2023-03-27T10:01:54.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.717+0000 7ff71f9db700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7ff7181509d0 0x7ff71806f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:54.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.717+0000 7ff71f9db700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff71806e4d0 0x7ff7180c6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:54.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.717+0000 7ff71f9db700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7181566c0 con 0x7ff7180c6210 2023-03-27T10:01:54.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.717+0000 7ff71f9db700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7181575c0 con 0x7ff7181509d0 2023-03-27T10:01:54.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.717+0000 7ff71f9db700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7180c5ea0 con 0x7ff71806e4d0 2023-03-27T10:01:54.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.717+0000 7ff71cf76700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7ff7180c6210 0x7ff71806ddc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:36176/0 (socket says 172.21.15.112:36176) 2023-03-27T10:01:54.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.717+0000 7ff71df78700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff71806e4d0 0x7ff7180c6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:54.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.717+0000 7ff71cf76700 1 -- 172.21.15.112:0/1921312209 learned_addr learned my addr 172.21.15.112:0/1921312209 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-03-27T10:01:54.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.717+0000 7ff70ffff700 1 -- 172.21.15.112:0/1921312209 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1153159209 0 0) 0x7ff7181566c0 con 0x7ff7180c6210 2023-03-27T10:01:54.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.717+0000 7ff70ffff700 1 -- 172.21.15.112:0/1921312209 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7040035b0 con 0x7ff7180c6210 2023-03-27T10:01:54.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.717+0000 7ff71d777700 1 --2- 172.21.15.112:0/1921312209 >> v2:172.21.15.23:3300/0 conn(0x7ff7181509d0 0x7ff71806f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:54.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.717+0000 7ff71df78700 1 -- 172.21.15.112:0/1921312209 >> v2:172.21.15.23:3300/0 conn(0x7ff7181509d0 msgr2=0x7ff71806f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:01:54.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.717+0000 7ff71df78700 1 --2- 172.21.15.112:0/1921312209 >> v2:172.21.15.23:3300/0 conn(0x7ff7181509d0 0x7ff71806f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:54.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.717+0000 7ff71df78700 1 -- 172.21.15.112:0/1921312209 >> v1:172.21.15.112:6789/0 conn(0x7ff7180c6210 legacy=0x7ff71806ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:54.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.717+0000 7ff71df78700 1 -- 172.21.15.112:0/1921312209 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff71815aab0 con 0x7ff71806e4d0 2023-03-27T10:01:54.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.717+0000 7ff71df78700 1 --2- 172.21.15.112:0/1921312209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff71806e4d0 0x7ff7180c6690 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff718072060 tx=0x7ff71000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17704b283cebcce7 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:01:54.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.717+0000 7ff70ffff700 1 -- 172.21.15.112:0/1921312209 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71000e070 con 0x7ff71806e4d0 2023-03-27T10:01:54.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.717+0000 7ff70ffff700 1 -- 172.21.15.112:0/1921312209 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff71000bb20 con 0x7ff71806e4d0 2023-03-27T10:01:54.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.717+0000 7ff70ffff700 1 -- 172.21.15.112:0/1921312209 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff710012430 con 0x7ff71806e4d0 2023-03-27T10:01:54.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.718+0000 7ff71f9db700 1 -- 172.21.15.112:0/1921312209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff71806e4d0 msgr2=0x7ff7180c6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:54.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.718+0000 7ff71f9db700 1 --2- 172.21.15.112:0/1921312209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff71806e4d0 0x7ff7180c6690 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff718072060 tx=0x7ff71000b330 comp rx=0 tx=0).stop 2023-03-27T10:01:54.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.718+0000 7ff71f9db700 1 -- 172.21.15.112:0/1921312209 shutdown_connections 2023-03-27T10:01:54.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.718+0000 7ff71f9db700 1 --2- 172.21.15.112:0/1921312209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff71806e4d0 0x7ff7180c6690 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:54.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.718+0000 7ff71f9db700 1 --2- 172.21.15.112:0/1921312209 >> v2:172.21.15.23:3300/0 conn(0x7ff7181509d0 0x7ff71806f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:54.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.718+0000 7ff71f9db700 1 -- 172.21.15.112:0/1921312209 >> 172.21.15.112:0/1921312209 conn(0x7ff718065ca0 msgr2=0x7ff718066070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:01:54.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.718+0000 7ff71f9db700 1 -- 172.21.15.112:0/1921312209 shutdown_connections 2023-03-27T10:01:54.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.718+0000 7ff71f9db700 1 -- 172.21.15.112:0/1921312209 wait complete. 2023-03-27T10:01:54.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.719+0000 7ff71f9db700 1 Processor -- start 2023-03-27T10:01:54.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.719+0000 7ff71f9db700 1 -- start start 2023-03-27T10:01:54.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.720+0000 7ff71f9db700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7ff71806e4d0 0x7ff7181ee0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:54.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.720+0000 7ff71f9db700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7180c6210 0x7ff718159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:54.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.720+0000 7ff71f9db700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff7181509d0 0x7ff7180cbe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:54.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.720+0000 7ff71f9db700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff718151dd0 con 0x7ff71806e4d0 2023-03-27T10:01:54.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.720+0000 7ff71f9db700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff71806c5b0 con 0x7ff7180c6210 2023-03-27T10:01:54.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.720+0000 7ff71f9db700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff71806c890 con 0x7ff7181509d0 2023-03-27T10:01:54.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.720+0000 7ff71cf76700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7180c6210 0x7ff718159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:54.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.720+0000 7ff71df78700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff7181509d0 0x7ff7180cbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:54.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.720+0000 7ff71cf76700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7180c6210 0x7ff718159640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36234/0 (socket says 172.21.15.112:36234) 2023-03-27T10:01:54.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.720+0000 7ff71cf76700 1 -- 172.21.15.112:0/2496756395 learned_addr learned my addr 172.21.15.112:0/2496756395 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-03-27T10:01:54.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.720+0000 7ff71d777700 1 --2- 172.21.15.112:0/2496756395 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7ff71806e4d0 0x7ff7181ee0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:54.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.720+0000 7ff71cf76700 1 -- 172.21.15.112:0/2496756395 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff7181509d0 msgr2=0x7ff7180cbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:54.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.720+0000 7ff71cf76700 1 --2- 172.21.15.112:0/2496756395 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff7181509d0 0x7ff7180cbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:54.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.721+0000 7ff71cf76700 1 -- 172.21.15.112:0/2496756395 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7ff71806e4d0 msgr2=0x7ff7181ee0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:54.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.721+0000 7ff71cf76700 1 --2- 172.21.15.112:0/2496756395 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7ff71806e4d0 0x7ff7181ee0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:54.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.721+0000 7ff71cf76700 1 -- 172.21.15.112:0/2496756395 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff710009a70 con 0x7ff7180c6210 2023-03-27T10:01:54.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.721+0000 7ff71d777700 1 --2- 172.21.15.112:0/2496756395 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7ff71806e4d0 0x7ff7181ee0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-27T10:01:54.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.721+0000 7ff71cf76700 1 --2- 172.21.15.112:0/2496756395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7180c6210 0x7ff718159640 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff708001eb0 tx=0x7ff70800c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:01:54.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.721+0000 7ff70dffb700 1 -- 172.21.15.112:0/2496756395 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff708005d70 con 0x7ff7180c6210 2023-03-27T10:01:54.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.721+0000 7ff70dffb700 1 -- 172.21.15.112:0/2496756395 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7080070d0 con 0x7ff7180c6210 2023-03-27T10:01:54.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.721+0000 7ff71f9db700 1 -- 172.21.15.112:0/2496756395 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff718156940 con 0x7ff7180c6210 2023-03-27T10:01:54.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.721+0000 7ff70dffb700 1 -- 172.21.15.112:0/2496756395 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff708007460 con 0x7ff7180c6210 2023-03-27T10:01:54.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.721+0000 7ff70dffb700 1 -- 172.21.15.112:0/2496756395 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7ff708019030 con 0x7ff7180c6210 2023-03-27T10:01:54.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.721+0000 7ff71f9db700 1 -- 172.21.15.112:0/2496756395 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff718150da0 con 0x7ff7180c6210 2023-03-27T10:01:54.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.721+0000 7ff70dffb700 1 -- 172.21.15.112:0/2496756395 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff7080075c0 con 0x7ff7180c6210 2023-03-27T10:01:54.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.722+0000 7ff71f9db700 1 -- 172.21.15.112:0/2496756395 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff718156c20 con 0x7ff7180c6210 2023-03-27T10:01:54.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.724+0000 7ff70dffb700 1 -- 172.21.15.112:0/2496756395 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72400 (secure 0 0 0) 0x7ff708017070 con 0x7ff7180c6210 2023-03-27T10:01:54.746 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-03-27T10:01:54.746+0000 7f105b615200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-03-27T10:01:54.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.808+0000 7ff71f9db700 1 -- 172.21.15.112:0/2496756395 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd new", "uuid": "df086052-536b-4fb1-bcce-7aaf5a13911e", "id": 0} v 0) v1 -- 0x7ff718069220 con 0x7ff7180c6210 2023-03-27T10:01:54.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.815+0000 7ff70dffb700 1 -- 172.21.15.112:0/2496756395 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "df086052-536b-4fb1-bcce-7aaf5a13911e", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff708007780 con 0x7ff7180c6210 2023-03-27T10:01:54.816 INFO:teuthology.orchestra.run.smithi112.stdout:0 2023-03-27T10:01:54.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.816+0000 7ff71f9db700 1 -- 172.21.15.112:0/2496756395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7180c6210 msgr2=0x7ff718159640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:54.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.816+0000 7ff71f9db700 1 --2- 172.21.15.112:0/2496756395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7180c6210 0x7ff718159640 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff708001eb0 tx=0x7ff70800c860 comp rx=0 tx=0).stop 2023-03-27T10:01:54.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.816+0000 7ff71f9db700 1 -- 172.21.15.112:0/2496756395 shutdown_connections 2023-03-27T10:01:54.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.816+0000 7ff71f9db700 1 --2- 172.21.15.112:0/2496756395 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff7181509d0 0x7ff7180cbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:54.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.816+0000 7ff71f9db700 1 --2- 172.21.15.112:0/2496756395 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7ff71806e4d0 0x7ff7181ee0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:54.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.816+0000 7ff71f9db700 1 --2- 172.21.15.112:0/2496756395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7180c6210 0x7ff718159640 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:54.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.816+0000 7ff71f9db700 1 -- 172.21.15.112:0/2496756395 >> 172.21.15.112:0/2496756395 conn(0x7ff718065ca0 msgr2=0x7ff7180c4640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:01:54.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.817+0000 7ff71f9db700 1 -- 172.21.15.112:0/2496756395 shutdown_connections 2023-03-27T10:01:54.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.817+0000 7ff71f9db700 1 -- 172.21.15.112:0/2496756395 wait complete. 2023-03-27T10:01:54.828 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new 03138001-a3fd-43fc-9797-a47643d9a3a1 1 2023-03-27T10:01:54.862 INFO:tasks.ceph.mgr.y.smithi023.stderr:2023-03-27T10:01:54.863+0000 7f6256375200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-03-27T10:01:54.885 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-03-27T10:01:54.885+0000 7f105b615200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-03-27T10:01:54.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.949+0000 7fc0a6eec700 1 Processor -- start 2023-03-27T10:01:54.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.950+0000 7fc0a6eec700 1 -- start start 2023-03-27T10:01:54.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.950+0000 7fc0a6eec700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc0a00c7320 0x7fc0a006f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:54.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.950+0000 7fc0a6eec700 1 -- --> v1:172.21.15.23:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0a006e820 con 0x7fc0a006b300 2023-03-27T10:01:54.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.950+0000 7fc0a6eec700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0a00c86a0 con 0x7fc0a00c6920 2023-03-27T10:01:54.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.950+0000 7fc0a6eec700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a0159690 con 0x7fc0a00c7320 2023-03-27T10:01:54.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.950+0000 7fc0a4c88700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fc0a00c6920 0x7fc0a00c7f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:36188/0 (socket says 172.21.15.112:36188) 2023-03-27T10:01:54.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.950+0000 7fc09ffff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc0a00c7320 0x7fc0a006f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:54.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.950+0000 7fc0a4c88700 1 -- 172.21.15.112:0/2583948116 learned_addr learned my addr 172.21.15.112:0/2583948116 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-03-27T10:01:54.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.951+0000 7fc09f7fe700 1 -- 172.21.15.112:0/2583948116 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3837520467 0 0) 0x7fc0a00c86a0 con 0x7fc0a00c6920 2023-03-27T10:01:54.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.951+0000 7fc09f7fe700 1 -- 172.21.15.112:0/2583948116 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0900035b0 con 0x7fc0a00c6920 2023-03-27T10:01:54.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.951+0000 7fc09ffff700 1 -- 172.21.15.112:0/2583948116 >> v1:172.21.15.112:6789/0 conn(0x7fc0a00c6920 legacy=0x7fc0a00c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:54.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.951+0000 7fc09ffff700 1 -- 172.21.15.112:0/2583948116 >> v1:172.21.15.23:6789/0 conn(0x7fc0a006b300 legacy=0x7fc0a0163150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:54.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.951+0000 7fc09ffff700 1 -- 172.21.15.112:0/2583948116 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a006dc10 con 0x7fc0a00c7320 2023-03-27T10:01:54.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.951+0000 7fc09ffff700 1 --2- 172.21.15.112:0/2583948116 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc0a00c7320 0x7fc0a006f200 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc08c009870 tx=0x7fc08c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffa9bf5343fce105 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:01:54.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.951+0000 7fc09f7fe700 1 -- 172.21.15.112:0/2583948116 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c011830 con 0x7fc0a00c7320 2023-03-27T10:01:54.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.951+0000 7fc09f7fe700 1 -- 172.21.15.112:0/2583948116 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc08c011990 con 0x7fc0a00c7320 2023-03-27T10:01:54.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.951+0000 7fc09f7fe700 1 -- 172.21.15.112:0/2583948116 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c011d20 con 0x7fc0a00c7320 2023-03-27T10:01:54.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.952+0000 7fc0a6eec700 1 -- 172.21.15.112:0/2583948116 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc0a00c7320 msgr2=0x7fc0a006f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:54.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.952+0000 7fc0a6eec700 1 --2- 172.21.15.112:0/2583948116 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc0a00c7320 0x7fc0a006f200 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc08c009870 tx=0x7fc08c009b80 comp rx=0 tx=0).stop 2023-03-27T10:01:54.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.952+0000 7fc0a6eec700 1 -- 172.21.15.112:0/2583948116 shutdown_connections 2023-03-27T10:01:54.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.952+0000 7fc0a6eec700 1 --2- 172.21.15.112:0/2583948116 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc0a00c7320 0x7fc0a006f200 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:54.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.952+0000 7fc0a6eec700 1 -- 172.21.15.112:0/2583948116 >> 172.21.15.112:0/2583948116 conn(0x7fc0a0065ca0 msgr2=0x7fc0a0066070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:01:54.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.952+0000 7fc0a6eec700 1 -- 172.21.15.112:0/2583948116 shutdown_connections 2023-03-27T10:01:54.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.952+0000 7fc0a6eec700 1 -- 172.21.15.112:0/2583948116 wait complete. 2023-03-27T10:01:54.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.953+0000 7fc0a6eec700 1 Processor -- start 2023-03-27T10:01:54.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.953+0000 7fc0a6eec700 1 -- start start 2023-03-27T10:01:54.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.954+0000 7fc0a6eec700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0a006b300 0x7fc0a0155840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:54.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.954+0000 7fc0a6eec700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc0a00c6920 0x7fc0a0155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:54.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.954+0000 7fc0a6eec700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fc0a00c7320 0x7fc0a01ef7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:54.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.954+0000 7fc0a6eec700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a00c38f0 con 0x7fc0a00c7320 2023-03-27T10:01:54.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.954+0000 7fc0a6eec700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a01ef310 con 0x7fc0a006b300 2023-03-27T10:01:54.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.954+0000 7fc0a6eec700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a00c3610 con 0x7fc0a00c6920 2023-03-27T10:01:54.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.954+0000 7fc09ffff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc0a00c6920 0x7fc0a0155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:54.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.954+0000 7fc09ffff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc0a00c6920 0x7fc0a0155d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.112:54280/0 (socket says 172.21.15.112:54280) 2023-03-27T10:01:54.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.954+0000 7fc09ffff700 1 -- 172.21.15.112:0/840416135 learned_addr learned my addr 172.21.15.112:0/840416135 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-03-27T10:01:54.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.954+0000 7fc0a4c88700 1 --2- 172.21.15.112:0/840416135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0a006b300 0x7fc0a0155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:54.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.954+0000 7fc0a5489700 1 --2- 172.21.15.112:0/840416135 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fc0a00c7320 0x7fc0a01ef7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:54.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.954+0000 7fc09ffff700 1 -- 172.21.15.112:0/840416135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0a006b300 msgr2=0x7fc0a0155840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:54.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.954+0000 7fc09ffff700 1 --2- 172.21.15.112:0/840416135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0a006b300 0x7fc0a0155840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:54.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.954+0000 7fc09ffff700 1 -- 172.21.15.112:0/840416135 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fc0a00c7320 msgr2=0x7fc0a01ef7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:54.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.954+0000 7fc09ffff700 1 --2- 172.21.15.112:0/840416135 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fc0a00c7320 0x7fc0a01ef7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:54.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.954+0000 7fc09ffff700 1 -- 172.21.15.112:0/840416135 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc08c0094a0 con 0x7fc0a00c6920 2023-03-27T10:01:54.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.954+0000 7fc0a4c88700 1 --2- 172.21.15.112:0/840416135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0a006b300 0x7fc0a0155840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-27T10:01:54.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.954+0000 7fc09ffff700 1 --2- 172.21.15.112:0/840416135 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc0a00c6920 0x7fc0a0155d80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc08c014040 tx=0x7fc08c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:01:54.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.955+0000 7fc09d7fa700 1 -- 172.21.15.112:0/840416135 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c0036a0 con 0x7fc0a00c6920 2023-03-27T10:01:54.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.955+0000 7fc09d7fa700 1 -- 172.21.15.112:0/840416135 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc08c01f070 con 0x7fc0a00c6920 2023-03-27T10:01:54.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.955+0000 7fc0a6eec700 1 -- 172.21.15.112:0/840416135 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0a006e1d0 con 0x7fc0a00c6920 2023-03-27T10:01:54.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.955+0000 7fc09d7fa700 1 -- 172.21.15.112:0/840416135 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c003a60 con 0x7fc0a00c6920 2023-03-27T10:01:54.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.955+0000 7fc0a6eec700 1 -- 172.21.15.112:0/840416135 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0a006def0 con 0x7fc0a00c6920 2023-03-27T10:01:54.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.955+0000 7fc09d7fa700 1 -- 172.21.15.112:0/840416135 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fc08c00f050 con 0x7fc0a00c6920 2023-03-27T10:01:54.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.955+0000 7fc09d7fa700 1 -- 172.21.15.112:0/840416135 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fc08c021070 con 0x7fc0a00c6920 2023-03-27T10:01:54.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.956+0000 7fc0a6eec700 1 -- 172.21.15.112:0/840416135 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0a006e4b0 con 0x7fc0a00c6920 2023-03-27T10:01:54.976 INFO:tasks.ceph.mgr.y.smithi023.stderr:2023-03-27T10:01:54.958+0000 7f6256375200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-03-27T10:01:54.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:54.958+0000 7fc09d7fa700 1 -- 172.21.15.112:0/840416135 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72400 (secure 0 0 0) 0x7fc08c015020 con 0x7fc0a00c6920 2023-03-27T10:01:54.977 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-03-27T10:01:54.969+0000 7f105b615200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-03-27T10:01:55.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.041+0000 7fc0a6eec700 1 -- 172.21.15.112:0/840416135 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd new", "uuid": "03138001-a3fd-43fc-9797-a47643d9a3a1", "id": 1} v 0) v1 -- 0x7fc0a0069220 con 0x7fc0a00c6920 2023-03-27T10:01:55.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.048+0000 7fc09d7fa700 1 -- 172.21.15.112:0/840416135 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "03138001-a3fd-43fc-9797-a47643d9a3a1", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc08c003c90 con 0x7fc0a00c6920 2023-03-27T10:01:55.049 INFO:teuthology.orchestra.run.smithi112.stdout:1 2023-03-27T10:01:55.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.050+0000 7fc0a6eec700 1 -- 172.21.15.112:0/840416135 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc0a00c6920 msgr2=0x7fc0a0155d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:55.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.050+0000 7fc0a6eec700 1 --2- 172.21.15.112:0/840416135 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc0a00c6920 0x7fc0a0155d80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc08c014040 tx=0x7fc08c004750 comp rx=0 tx=0).stop 2023-03-27T10:01:55.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.050+0000 7fc0a6eec700 1 -- 172.21.15.112:0/840416135 shutdown_connections 2023-03-27T10:01:55.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.050+0000 7fc0a6eec700 1 --2- 172.21.15.112:0/840416135 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fc0a00c7320 0x7fc0a01ef7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:55.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.050+0000 7fc0a6eec700 1 --2- 172.21.15.112:0/840416135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0a006b300 0x7fc0a0155840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:55.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.050+0000 7fc0a6eec700 1 --2- 172.21.15.112:0/840416135 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc0a00c6920 0x7fc0a0155d80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:55.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.050+0000 7fc0a6eec700 1 -- 172.21.15.112:0/840416135 >> 172.21.15.112:0/840416135 conn(0x7fc0a0065ca0 msgr2=0x7fc0a00cb950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:01:55.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.050+0000 7fc0a6eec700 1 -- 172.21.15.112:0/840416135 shutdown_connections 2023-03-27T10:01:55.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.050+0000 7fc0a6eec700 1 -- 172.21.15.112:0/840416135 wait complete. 2023-03-27T10:01:55.061 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new 964dd926-1160-44ef-aa34-b18965dc05f3 2 2023-03-27T10:01:55.107 INFO:tasks.ceph.mgr.y.smithi023.stderr:2023-03-27T10:01:55.107+0000 7f6256375200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-03-27T10:01:55.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.185+0000 7f5046e88700 1 Processor -- start 2023-03-27T10:01:55.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.185+0000 7f5046e88700 1 -- start start 2023-03-27T10:01:55.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.186+0000 7f5046e88700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f50400c4bf0 0x7f50400c4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:55.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.186+0000 7f5046e88700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f50401573d0 0x7f5040070af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:55.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.186+0000 7f5046e88700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50400c5aa0 con 0x7f5040156fa0 2023-03-27T10:01:55.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.186+0000 7f5046e88700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50401577d0 con 0x7f50400c4bf0 2023-03-27T10:01:55.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.186+0000 7f5046e88700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5040072840 con 0x7f50401573d0 2023-03-27T10:01:55.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.186+0000 7f5045425700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f50401573d0 0x7f5040070af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:55.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.186+0000 7f5045425700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f50401573d0 0x7f5040070af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36254/0 (socket says 172.21.15.112:36254) 2023-03-27T10:01:55.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.186+0000 7f5045425700 1 -- 172.21.15.112:0/334892005 learned_addr learned my addr 172.21.15.112:0/334892005 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-03-27T10:01:55.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.186+0000 7f5044c24700 1 --2- 172.21.15.112:0/334892005 >> v2:172.21.15.23:3300/0 conn(0x7f50400c4bf0 0x7f50400c4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:55.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.186+0000 7f5045425700 1 -- 172.21.15.112:0/334892005 >> v2:172.21.15.23:3300/0 conn(0x7f50400c4bf0 msgr2=0x7f50400c4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:01:55.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.186+0000 7f5045425700 1 --2- 172.21.15.112:0/334892005 >> v2:172.21.15.23:3300/0 conn(0x7f50400c4bf0 0x7f50400c4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:55.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.186+0000 7f5045425700 1 -- 172.21.15.112:0/334892005 >> v1:172.21.15.112:6789/0 conn(0x7f5040156fa0 legacy=0x7f504006eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:55.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.186+0000 7f5045425700 1 -- 172.21.15.112:0/334892005 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50400c5d40 con 0x7f50401573d0 2023-03-27T10:01:55.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.186+0000 7f5045425700 1 --2- 172.21.15.112:0/334892005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50401573d0 0x7f5040070af0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5040070110 tx=0x7f503800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=292afa94e4a8624 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:01:55.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.186+0000 7f503f7fe700 1 -- 172.21.15.112:0/334892005 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503800e070 con 0x7f50401573d0 2023-03-27T10:01:55.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.186+0000 7f503f7fe700 1 -- 172.21.15.112:0/334892005 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f503800b9c0 con 0x7f50401573d0 2023-03-27T10:01:55.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.186+0000 7f503f7fe700 1 -- 172.21.15.112:0/334892005 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503800bd70 con 0x7f50401573d0 2023-03-27T10:01:55.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.187+0000 7f5046e88700 1 -- 172.21.15.112:0/334892005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50401573d0 msgr2=0x7f5040070af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:55.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.187+0000 7f5046e88700 1 --2- 172.21.15.112:0/334892005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50401573d0 0x7f5040070af0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5040070110 tx=0x7f503800b330 comp rx=0 tx=0).stop 2023-03-27T10:01:55.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.187+0000 7f5046e88700 1 -- 172.21.15.112:0/334892005 shutdown_connections 2023-03-27T10:01:55.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.187+0000 7f5046e88700 1 --2- 172.21.15.112:0/334892005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50401573d0 0x7f5040070af0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:55.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.187+0000 7f5046e88700 1 --2- 172.21.15.112:0/334892005 >> v2:172.21.15.23:3300/0 conn(0x7f50400c4bf0 0x7f50400c4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:55.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.187+0000 7f5046e88700 1 -- 172.21.15.112:0/334892005 >> 172.21.15.112:0/334892005 conn(0x7f50400c8720 msgr2=0x7f504006e690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:01:55.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.187+0000 7f5046e88700 1 -- 172.21.15.112:0/334892005 shutdown_connections 2023-03-27T10:01:55.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.187+0000 7f5046e88700 1 -- 172.21.15.112:0/334892005 wait complete. 2023-03-27T10:01:55.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.188+0000 7f5046e88700 1 Processor -- start 2023-03-27T10:01:55.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.188+0000 7f5046e88700 1 -- start start 2023-03-27T10:01:55.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.189+0000 7f5046e88700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f50400c4bf0 0x7f504006a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:55.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.189+0000 7f5046e88700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5040156fa0 0x7f50400ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:55.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.189+0000 7f5046e88700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f50401573d0 0x7f50400ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:55.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.189+0000 7f5046e88700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5040151af0 con 0x7f50400c4bf0 2023-03-27T10:01:55.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.189+0000 7f5046e88700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5040151570 con 0x7f5040156fa0 2023-03-27T10:01:55.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.189+0000 7f5046e88700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50401ec630 con 0x7f50401573d0 2023-03-27T10:01:55.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.189+0000 7f503ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5040156fa0 0x7f50400ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:55.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.189+0000 7f5045425700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f50401573d0 0x7f50400ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:55.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.189+0000 7f503ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5040156fa0 0x7f50400ca1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36258/0 (socket says 172.21.15.112:36258) 2023-03-27T10:01:55.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.189+0000 7f503ffff700 1 -- 172.21.15.112:0/3827916354 learned_addr learned my addr 172.21.15.112:0/3827916354 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-03-27T10:01:55.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.189+0000 7f5044c24700 1 --2- 172.21.15.112:0/3827916354 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f50400c4bf0 0x7f504006a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:55.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.189+0000 7f5045425700 1 -- 172.21.15.112:0/3827916354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5040156fa0 msgr2=0x7f50400ca1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:55.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.189+0000 7f5045425700 1 --2- 172.21.15.112:0/3827916354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5040156fa0 0x7f50400ca1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:55.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.189+0000 7f5045425700 1 -- 172.21.15.112:0/3827916354 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f50400c4bf0 msgr2=0x7f504006a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:55.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.189+0000 7f5045425700 1 --2- 172.21.15.112:0/3827916354 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f50400c4bf0 0x7f504006a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:55.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.189+0000 7f5045425700 1 -- 172.21.15.112:0/3827916354 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5038009b30 con 0x7f50401573d0 2023-03-27T10:01:55.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.189+0000 7f503ffff700 1 --2- 172.21.15.112:0/3827916354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5040156fa0 0x7f50400ca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-27T10:01:55.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.190+0000 7f5045425700 1 --2- 172.21.15.112:0/3827916354 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f50401573d0 0x7f50400ca700 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5038015000 tx=0x7f5038004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:01:55.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.190+0000 7f503d7fa700 1 -- 172.21.15.112:0/3827916354 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5038004880 con 0x7f50401573d0 2023-03-27T10:01:55.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.190+0000 7f503d7fa700 1 -- 172.21.15.112:0/3827916354 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50380099b0 con 0x7f50401573d0 2023-03-27T10:01:55.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.190+0000 7f5046e88700 1 -- 172.21.15.112:0/3827916354 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50400c5fe0 con 0x7f50401573d0 2023-03-27T10:01:55.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.190+0000 7f503d7fa700 1 -- 172.21.15.112:0/3827916354 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503801c680 con 0x7f50401573d0 2023-03-27T10:01:55.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.190+0000 7f5046e88700 1 -- 172.21.15.112:0/3827916354 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50401567c0 con 0x7f50401573d0 2023-03-27T10:01:55.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.190+0000 7f503d7fa700 1 -- 172.21.15.112:0/3827916354 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f503801c7e0 con 0x7f50401573d0 2023-03-27T10:01:55.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.190+0000 7f503d7fa700 1 -- 172.21.15.112:0/3827916354 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f5038027830 con 0x7f50401573d0 2023-03-27T10:01:55.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.191+0000 7f5046e88700 1 -- 172.21.15.112:0/3827916354 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50400c62c0 con 0x7f50401573d0 2023-03-27T10:01:55.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.193+0000 7f503d7fa700 1 -- 172.21.15.112:0/3827916354 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72400 (secure 0 0 0) 0x7f503801cc50 con 0x7f50401573d0 2023-03-27T10:01:55.208 INFO:tasks.ceph.mgr.y.smithi023.stderr:2023-03-27T10:01:55.209+0000 7f6256375200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-03-27T10:01:55.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.275+0000 7f5046e88700 1 -- 172.21.15.112:0/3827916354 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd new", "uuid": "964dd926-1160-44ef-aa34-b18965dc05f3", "id": 2} v 0) v1 -- 0x7f5040069220 con 0x7f50401573d0 2023-03-27T10:01:55.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.287+0000 7f503d7fa700 1 -- 172.21.15.112:0/3827916354 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "964dd926-1160-44ef-aa34-b18965dc05f3", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5038027410 con 0x7f50401573d0 2023-03-27T10:01:55.288 INFO:teuthology.orchestra.run.smithi112.stdout:2 2023-03-27T10:01:55.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.288+0000 7f5046e88700 1 -- 172.21.15.112:0/3827916354 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f50401573d0 msgr2=0x7f50400ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:55.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.288+0000 7f5046e88700 1 --2- 172.21.15.112:0/3827916354 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f50401573d0 0x7f50400ca700 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5038015000 tx=0x7f5038004ad0 comp rx=0 tx=0).stop 2023-03-27T10:01:55.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.288+0000 7f5046e88700 1 -- 172.21.15.112:0/3827916354 shutdown_connections 2023-03-27T10:01:55.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.288+0000 7f5046e88700 1 --2- 172.21.15.112:0/3827916354 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f50401573d0 0x7f50400ca700 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:55.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.288+0000 7f5046e88700 1 --2- 172.21.15.112:0/3827916354 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f50400c4bf0 0x7f504006a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:55.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.288+0000 7f5046e88700 1 --2- 172.21.15.112:0/3827916354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5040156fa0 0x7f50400ca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:55.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.288+0000 7f5046e88700 1 -- 172.21.15.112:0/3827916354 >> 172.21.15.112:0/3827916354 conn(0x7f50400c8720 msgr2=0x7f504006e620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:01:55.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.288+0000 7f5046e88700 1 -- 172.21.15.112:0/3827916354 shutdown_connections 2023-03-27T10:01:55.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.288+0000 7f5046e88700 1 -- 172.21.15.112:0/3827916354 wait complete. 2023-03-27T10:01:55.302 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new 4ae5500f-cb24-44a8-a536-5c399c09e566 3 2023-03-27T10:01:55.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.423+0000 7f25d59a2700 1 Processor -- start 2023-03-27T10:01:55.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.423+0000 7f25d59a2700 1 -- start start 2023-03-27T10:01:55.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.424+0000 7f25d59a2700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25d0071530 0x7f25d00719b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:55.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.424+0000 7f25d59a2700 1 -- --> v1:172.21.15.23:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25d0071ef0 con 0x7f25d00c7f80 2023-03-27T10:01:55.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.424+0000 7f25d59a2700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25d0159730 con 0x7f25d0152c40 2023-03-27T10:01:55.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.424+0000 7f25d59a2700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d00c86b0 con 0x7f25d0071530 2023-03-27T10:01:55.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.424+0000 7f25ceffd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f25d0152c40 0x7f25d015c940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:36212/0 (socket says 172.21.15.112:36212) 2023-03-27T10:01:55.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.424+0000 7f25ceffd700 1 -- 172.21.15.112:0/1319408663 learned_addr learned my addr 172.21.15.112:0/1319408663 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-03-27T10:01:55.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.424+0000 7f25cf7fe700 1 --2- 172.21.15.112:0/1319408663 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25d0071530 0x7f25d00719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:55.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.424+0000 7f25cdffb700 1 -- 172.21.15.112:0/1319408663 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1112846431 0 0) 0x7f25d0159730 con 0x7f25d0152c40 2023-03-27T10:01:55.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.424+0000 7f25cdffb700 1 -- 172.21.15.112:0/1319408663 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25b80035b0 con 0x7f25d0152c40 2023-03-27T10:01:55.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.424+0000 7f25cf7fe700 1 -- 172.21.15.112:0/1319408663 >> v1:172.21.15.112:6789/0 conn(0x7f25d0152c40 legacy=0x7f25d015c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:55.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.424+0000 7f25cf7fe700 1 -- 172.21.15.112:0/1319408663 >> v1:172.21.15.23:6789/0 conn(0x7f25d00c7f80 legacy=0x7f25d006f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:55.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.424+0000 7f25cf7fe700 1 -- 172.21.15.112:0/1319408663 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d0158b20 con 0x7f25d0071530 2023-03-27T10:01:55.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.425+0000 7f25cf7fe700 1 --2- 172.21.15.112:0/1319408663 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25d0071530 0x7f25d00719b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f25c8009dc0 tx=0x7f25c800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba639dd2da91645f server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:01:55.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.425+0000 7f25cdffb700 1 -- 172.21.15.112:0/1319408663 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c800e070 con 0x7f25d0071530 2023-03-27T10:01:55.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.425+0000 7f25cdffb700 1 -- 172.21.15.112:0/1319408663 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c800bb20 con 0x7f25d0071530 2023-03-27T10:01:55.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.425+0000 7f25cdffb700 1 -- 172.21.15.112:0/1319408663 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c8012430 con 0x7f25d0071530 2023-03-27T10:01:55.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.425+0000 7f25d59a2700 1 -- 172.21.15.112:0/1319408663 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25d0071530 msgr2=0x7f25d00719b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:55.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.425+0000 7f25d59a2700 1 --2- 172.21.15.112:0/1319408663 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25d0071530 0x7f25d00719b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f25c8009dc0 tx=0x7f25c800b330 comp rx=0 tx=0).stop 2023-03-27T10:01:55.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.425+0000 7f25d59a2700 1 -- 172.21.15.112:0/1319408663 shutdown_connections 2023-03-27T10:01:55.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.425+0000 7f25d59a2700 1 --2- 172.21.15.112:0/1319408663 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25d0071530 0x7f25d00719b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:55.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.425+0000 7f25d59a2700 1 -- 172.21.15.112:0/1319408663 >> 172.21.15.112:0/1319408663 conn(0x7f25d00660c0 msgr2=0x7f25d006d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:01:55.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.425+0000 7f25d59a2700 1 -- 172.21.15.112:0/1319408663 shutdown_connections 2023-03-27T10:01:55.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.425+0000 7f25d59a2700 1 -- 172.21.15.112:0/1319408663 wait complete. 2023-03-27T10:01:55.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.427+0000 7f25d59a2700 1 Processor -- start 2023-03-27T10:01:55.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.427+0000 7f25d59a2700 1 -- start start 2023-03-27T10:01:55.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.427+0000 7f25d59a2700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f25d0071530 0x7f25d015ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:55.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.427+0000 7f25d59a2700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25d00c7f80 0x7f25d01f2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:55.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.427+0000 7f25d59a2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25d0152c40 0x7f25d00cae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:55.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.427+0000 7f25d59a2700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d015b280 con 0x7f25d0071530 2023-03-27T10:01:55.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.427+0000 7f25d59a2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d0154300 con 0x7f25d0152c40 2023-03-27T10:01:55.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.427+0000 7f25d59a2700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d01f2760 con 0x7f25d00c7f80 2023-03-27T10:01:55.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.428+0000 7f25ce7fc700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25d00c7f80 0x7f25d01f2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:55.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.428+0000 7f25cf7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25d0152c40 0x7f25d00cae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:55.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.428+0000 7f25ce7fc700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25d00c7f80 0x7f25d01f2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.112:54304/0 (socket says 172.21.15.112:54304) 2023-03-27T10:01:55.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.428+0000 7f25cf7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25d0152c40 0x7f25d00cae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36272/0 (socket says 172.21.15.112:36272) 2023-03-27T10:01:55.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.428+0000 7f25ce7fc700 1 -- 172.21.15.112:0/1871374242 learned_addr learned my addr 172.21.15.112:0/1871374242 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-03-27T10:01:55.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.428+0000 7f25ceffd700 1 --2- 172.21.15.112:0/1871374242 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f25d0071530 0x7f25d015ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:55.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.428+0000 7f25cf7fe700 1 -- 172.21.15.112:0/1871374242 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25d00c7f80 msgr2=0x7f25d01f2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:55.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.428+0000 7f25cf7fe700 1 --2- 172.21.15.112:0/1871374242 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25d00c7f80 0x7f25d01f2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:55.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.428+0000 7f25cf7fe700 1 -- 172.21.15.112:0/1871374242 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f25d0071530 msgr2=0x7f25d015ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:55.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.428+0000 7f25cf7fe700 1 --2- 172.21.15.112:0/1871374242 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f25d0071530 0x7f25d015ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:55.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.428+0000 7f25cf7fe700 1 -- 172.21.15.112:0/1871374242 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c8009a70 con 0x7f25d0152c40 2023-03-27T10:01:55.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.428+0000 7f25ce7fc700 1 --2- 172.21.15.112:0/1871374242 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25d00c7f80 0x7f25d01f2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-27T10:01:55.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.428+0000 7f25cf7fe700 1 --2- 172.21.15.112:0/1871374242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25d0152c40 0x7f25d00cae30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f25d006fae0 tx=0x7f25c8004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:01:55.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.428+0000 7f25b7fff700 1 -- 172.21.15.112:0/1871374242 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c8012430 con 0x7f25d0152c40 2023-03-27T10:01:55.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.428+0000 7f25b7fff700 1 -- 172.21.15.112:0/1871374242 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c8020070 con 0x7f25d0152c40 2023-03-27T10:01:55.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.428+0000 7f25b7fff700 1 -- 172.21.15.112:0/1871374242 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c8004280 con 0x7f25d0152c40 2023-03-27T10:01:55.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.428+0000 7f25d59a2700 1 -- 172.21.15.112:0/1871374242 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25d01590e0 con 0x7f25d0152c40 2023-03-27T10:01:55.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.428+0000 7f25d59a2700 1 -- 172.21.15.112:0/1871374242 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25d0158e00 con 0x7f25d0152c40 2023-03-27T10:01:55.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.428+0000 7f25b7fff700 1 -- 172.21.15.112:0/1871374242 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f25c80046d0 con 0x7f25d0152c40 2023-03-27T10:01:55.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.428+0000 7f25b7fff700 1 -- 172.21.15.112:0/1871374242 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f25c8022070 con 0x7f25d0152c40 2023-03-27T10:01:55.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.429+0000 7f25d59a2700 1 -- 172.21.15.112:0/1871374242 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25d01593c0 con 0x7f25d0152c40 2023-03-27T10:01:55.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.431+0000 7f25b7fff700 1 -- 172.21.15.112:0/1871374242 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72400 (secure 0 0 0) 0x7f25c802c060 con 0x7f25d0152c40 2023-03-27T10:01:55.446 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-03-27T10:01:55.437+0000 7f105b615200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-03-27T10:01:55.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.515+0000 7f25d59a2700 1 -- 172.21.15.112:0/1871374242 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4ae5500f-cb24-44a8-a536-5c399c09e566", "id": 3} v 0) v1 -- 0x7f25d0069220 con 0x7f25d0152c40 2023-03-27T10:01:55.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.519+0000 7f25b7fff700 1 -- 172.21.15.112:0/1871374242 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4ae5500f-cb24-44a8-a536-5c399c09e566", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f25c8027480 con 0x7f25d0152c40 2023-03-27T10:01:55.519 INFO:teuthology.orchestra.run.smithi112.stdout:3 2023-03-27T10:01:55.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.520+0000 7f25d59a2700 1 -- 172.21.15.112:0/1871374242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25d0152c40 msgr2=0x7f25d00cae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:55.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.520+0000 7f25d59a2700 1 --2- 172.21.15.112:0/1871374242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25d0152c40 0x7f25d00cae30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f25d006fae0 tx=0x7f25c8004ad0 comp rx=0 tx=0).stop 2023-03-27T10:01:55.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.520+0000 7f25d59a2700 1 -- 172.21.15.112:0/1871374242 shutdown_connections 2023-03-27T10:01:55.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.520+0000 7f25d59a2700 1 --2- 172.21.15.112:0/1871374242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25d0152c40 0x7f25d00cae30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:55.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.520+0000 7f25d59a2700 1 --2- 172.21.15.112:0/1871374242 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f25d0071530 0x7f25d015ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:55.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.520+0000 7f25d59a2700 1 --2- 172.21.15.112:0/1871374242 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25d00c7f80 0x7f25d01f2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:55.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.520+0000 7f25d59a2700 1 -- 172.21.15.112:0/1871374242 >> 172.21.15.112:0/1871374242 conn(0x7f25d00660c0 msgr2=0x7f25d006d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:01:55.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.520+0000 7f25d59a2700 1 -- 172.21.15.112:0/1871374242 shutdown_connections 2023-03-27T10:01:55.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.520+0000 7f25d59a2700 1 -- 172.21.15.112:0/1871374242 wait complete. 2023-03-27T10:01:55.533 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new de228ab2-92b4-40bc-8dbd-8f2f4895dc22 4 2023-03-27T10:01:55.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.653+0000 7f3c3e8e5700 1 Processor -- start 2023-03-27T10:01:55.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.653+0000 7f3c3e8e5700 1 -- start start 2023-03-27T10:01:55.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.655+0000 7f3c3e8e5700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3c380c4fc0 0x7f3c3806eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:55.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.655+0000 7f3c3e8e5700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3c3806f2e0 0x7f3c38070af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:55.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.655+0000 7f3c3e8e5700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c380c5aa0 con 0x7f3c380c4bf0 2023-03-27T10:01:55.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.655+0000 7f3c3e8e5700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c38072840 con 0x7f3c380c4fc0 2023-03-27T10:01:55.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.655+0000 7f3c3e8e5700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c380c5d40 con 0x7f3c3806f2e0 2023-03-27T10:01:55.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.655+0000 7f3c377fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3c380c4fc0 0x7f3c3806eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:55.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.655+0000 7f3c3ce82700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3c3806f2e0 0x7f3c38070af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:55.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.655+0000 7f3c377fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3c380c4fc0 0x7f3c3806eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36276/0 (socket says 172.21.15.112:36276) 2023-03-27T10:01:55.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.655+0000 7f3c377fe700 1 -- 172.21.15.112:0/4204996253 learned_addr learned my addr 172.21.15.112:0/4204996253 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-03-27T10:01:55.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.655+0000 7f3c377fe700 1 -- 172.21.15.112:0/4204996253 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3c3806f2e0 msgr2=0x7f3c38070af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:01:55.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.655+0000 7f3c377fe700 1 --2- 172.21.15.112:0/4204996253 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3c3806f2e0 0x7f3c38070af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:55.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.655+0000 7f3c377fe700 1 -- 172.21.15.112:0/4204996253 >> v1:172.21.15.112:6789/0 conn(0x7f3c380c4bf0 legacy=0x7f3c38156fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:55.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.655+0000 7f3c377fe700 1 -- 172.21.15.112:0/4204996253 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c380c53f0 con 0x7f3c380c4fc0 2023-03-27T10:01:55.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.655+0000 7f3c3ce82700 1 --2- 172.21.15.112:0/4204996253 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3c3806f2e0 0x7f3c38070af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-27T10:01:55.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.655+0000 7f3c377fe700 1 --2- 172.21.15.112:0/4204996253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c380c4fc0 0x7f3c3806eda0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c0098f0 tx=0x7f3c2c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26a94208c9b81ae9 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:01:55.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.655+0000 7f3c36ffd700 1 -- 172.21.15.112:0/4204996253 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c011830 con 0x7f3c380c4fc0 2023-03-27T10:01:55.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.656+0000 7f3c36ffd700 1 -- 172.21.15.112:0/4204996253 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c2c011990 con 0x7f3c380c4fc0 2023-03-27T10:01:55.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.656+0000 7f3c36ffd700 1 -- 172.21.15.112:0/4204996253 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c011d20 con 0x7f3c380c4fc0 2023-03-27T10:01:55.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.656+0000 7f3c3e8e5700 1 -- 172.21.15.112:0/4204996253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c380c4fc0 msgr2=0x7f3c3806eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:55.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.656+0000 7f3c3e8e5700 1 --2- 172.21.15.112:0/4204996253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c380c4fc0 0x7f3c3806eda0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c0098f0 tx=0x7f3c2c009c00 comp rx=0 tx=0).stop 2023-03-27T10:01:55.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.656+0000 7f3c3e8e5700 1 -- 172.21.15.112:0/4204996253 shutdown_connections 2023-03-27T10:01:55.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.656+0000 7f3c3e8e5700 1 --2- 172.21.15.112:0/4204996253 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3c3806f2e0 0x7f3c38070af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:55.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.656+0000 7f3c3e8e5700 1 --2- 172.21.15.112:0/4204996253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c380c4fc0 0x7f3c3806eda0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:55.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.656+0000 7f3c3e8e5700 1 -- 172.21.15.112:0/4204996253 >> 172.21.15.112:0/4204996253 conn(0x7f3c380c8720 msgr2=0x7f3c3806e690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:01:55.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.656+0000 7f3c3e8e5700 1 -- 172.21.15.112:0/4204996253 shutdown_connections 2023-03-27T10:01:55.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.656+0000 7f3c3e8e5700 1 -- 172.21.15.112:0/4204996253 wait complete. 2023-03-27T10:01:55.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.658+0000 7f3c3e8e5700 1 Processor -- start 2023-03-27T10:01:55.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.658+0000 7f3c3e8e5700 1 -- start start 2023-03-27T10:01:55.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.659+0000 7f3c3e8e5700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3c3806f2e0 0x7f3c3806a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:55.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.659+0000 7f3c3e8e5700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f3c380c4bf0 0x7f3c381ef480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:55.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.660+0000 7f3c3e8e5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c380c4fc0 0x7f3c381ef9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:55.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.660+0000 7f3c3e8e5700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c38138d50 con 0x7f3c380c4bf0 2023-03-27T10:01:55.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.660+0000 7f3c3e8e5700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c38139350 con 0x7f3c380c4fc0 2023-03-27T10:01:55.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.660+0000 7f3c3e8e5700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c381efff0 con 0x7f3c3806f2e0 2023-03-27T10:01:55.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.660+0000 7f3c3ce82700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c380c4fc0 0x7f3c381ef9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:55.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.660+0000 7f3c37fff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3c3806f2e0 0x7f3c3806a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:55.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.660+0000 7f3c3ce82700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c380c4fc0 0x7f3c381ef9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36284/0 (socket says 172.21.15.112:36284) 2023-03-27T10:01:55.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.660+0000 7f3c37fff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3c3806f2e0 0x7f3c3806a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.112:54314/0 (socket says 172.21.15.112:54314) 2023-03-27T10:01:55.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.660+0000 7f3c3ce82700 1 -- 172.21.15.112:0/617906644 learned_addr learned my addr 172.21.15.112:0/617906644 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-03-27T10:01:55.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.660+0000 7f3c377fe700 1 --2- 172.21.15.112:0/617906644 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f3c380c4bf0 0x7f3c381ef480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:55.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.660+0000 7f3c3ce82700 1 -- 172.21.15.112:0/617906644 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3c3806f2e0 msgr2=0x7f3c3806a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:55.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.660+0000 7f3c3ce82700 1 --2- 172.21.15.112:0/617906644 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3c3806f2e0 0x7f3c3806a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:55.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.660+0000 7f3c3ce82700 1 -- 172.21.15.112:0/617906644 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f3c380c4bf0 msgr2=0x7f3c381ef480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:55.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.660+0000 7f3c3ce82700 1 --2- 172.21.15.112:0/617906644 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f3c380c4bf0 0x7f3c381ef480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:55.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.660+0000 7f3c3ce82700 1 -- 172.21.15.112:0/617906644 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c30009a70 con 0x7f3c380c4fc0 2023-03-27T10:01:55.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.660+0000 7f3c37fff700 1 --2- 172.21.15.112:0/617906644 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3c3806f2e0 0x7f3c3806a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-27T10:01:55.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.660+0000 7f3c3ce82700 1 --2- 172.21.15.112:0/617906644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c380c4fc0 0x7f3c381ef9c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3c3000efc0 tx=0x7f3c3000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:01:55.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.660+0000 7f3c34ff9700 1 -- 172.21.15.112:0/617906644 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3000cb40 con 0x7f3c380c4fc0 2023-03-27T10:01:55.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.660+0000 7f3c34ff9700 1 -- 172.21.15.112:0/617906644 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c3000f040 con 0x7f3c380c4fc0 2023-03-27T10:01:55.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.660+0000 7f3c34ff9700 1 -- 172.21.15.112:0/617906644 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c30003a20 con 0x7f3c380c4fc0 2023-03-27T10:01:55.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.660+0000 7f3c3e8e5700 1 -- 172.21.15.112:0/617906644 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c2c009550 con 0x7f3c380c4fc0 2023-03-27T10:01:55.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.660+0000 7f3c3e8e5700 1 -- 172.21.15.112:0/617906644 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c380c5fe0 con 0x7f3c380c4fc0 2023-03-27T10:01:55.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.661+0000 7f3c34ff9700 1 -- 172.21.15.112:0/617906644 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f3c30014480 con 0x7f3c380c4fc0 2023-03-27T10:01:55.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.661+0000 7f3c34ff9700 1 -- 172.21.15.112:0/617906644 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f3c3001d970 con 0x7f3c380c4fc0 2023-03-27T10:01:55.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.661+0000 7f3c3e8e5700 1 -- 172.21.15.112:0/617906644 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c381567c0 con 0x7f3c380c4fc0 2023-03-27T10:01:55.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.665+0000 7f3c34ff9700 1 -- 172.21.15.112:0/617906644 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72400 (secure 0 0 0) 0x7f3c30018030 con 0x7f3c380c4fc0 2023-03-27T10:01:55.732 INFO:tasks.ceph.mgr.y.smithi023.stderr:2023-03-27T10:01:55.732+0000 7f6256375200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-03-27T10:01:55.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.748+0000 7f3c3e8e5700 1 -- 172.21.15.112:0/617906644 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd new", "uuid": "de228ab2-92b4-40bc-8dbd-8f2f4895dc22", "id": 4} v 0) v1 -- 0x7f3c380c62c0 con 0x7f3c380c4fc0 2023-03-27T10:01:55.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.752+0000 7f3c34ff9700 1 -- 172.21.15.112:0/617906644 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "de228ab2-92b4-40bc-8dbd-8f2f4895dc22", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3c3001dbe0 con 0x7f3c380c4fc0 2023-03-27T10:01:55.752 INFO:teuthology.orchestra.run.smithi112.stdout:4 2023-03-27T10:01:55.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.753+0000 7f3c3e8e5700 1 -- 172.21.15.112:0/617906644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c380c4fc0 msgr2=0x7f3c381ef9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:55.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.753+0000 7f3c3e8e5700 1 --2- 172.21.15.112:0/617906644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c380c4fc0 0x7f3c381ef9c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3c3000efc0 tx=0x7f3c3000c3a0 comp rx=0 tx=0).stop 2023-03-27T10:01:55.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.753+0000 7f3c3e8e5700 1 -- 172.21.15.112:0/617906644 shutdown_connections 2023-03-27T10:01:55.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.753+0000 7f3c3e8e5700 1 --2- 172.21.15.112:0/617906644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c380c4fc0 0x7f3c381ef9c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:55.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.753+0000 7f3c3e8e5700 1 --2- 172.21.15.112:0/617906644 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3c3806f2e0 0x7f3c3806a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:55.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.753+0000 7f3c3e8e5700 1 --2- 172.21.15.112:0/617906644 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f3c380c4bf0 0x7f3c381ef480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:55.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.753+0000 7f3c3e8e5700 1 -- 172.21.15.112:0/617906644 >> 172.21.15.112:0/617906644 conn(0x7f3c380c8720 msgr2=0x7f3c38154dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:01:55.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.753+0000 7f3c3e8e5700 1 -- 172.21.15.112:0/617906644 shutdown_connections 2023-03-27T10:01:55.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.753+0000 7f3c3e8e5700 1 -- 172.21.15.112:0/617906644 wait complete. 2023-03-27T10:01:55.764 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new 6c839571-b490-4e66-92ac-0d7301c2e677 5 2023-03-27T10:01:55.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.910+0000 7f3cf2fba700 1 Processor -- start 2023-03-27T10:01:55.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.910+0000 7f3cf2fba700 1 -- start start 2023-03-27T10:01:55.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.911+0000 7f3cf2fba700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3cec14eff0 0x7f3cec155780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:55.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.911+0000 7f3cf2fba700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3cec0b07f0 0x7f3cec071e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:55.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.911+0000 7f3cf2fba700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cec070b90 con 0x7f3cec0723d0 2023-03-27T10:01:55.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.911+0000 7f3cf2fba700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cec0c5b30 con 0x7f3cec14eff0 2023-03-27T10:01:55.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.911+0000 7f3cf2fba700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cec070e00 con 0x7f3cec0b07f0 2023-03-27T10:01:55.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.911+0000 7f3cebfff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3cec0b07f0 0x7f3cec071e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:55.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.911+0000 7f3cf0d56700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3cec14eff0 0x7f3cec155780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:55.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.911+0000 7f3cebfff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3cec0b07f0 0x7f3cec071e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.112:54322/0 (socket says 172.21.15.112:54322) 2023-03-27T10:01:55.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.911+0000 7f3cf0d56700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3cec14eff0 0x7f3cec155780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36286/0 (socket says 172.21.15.112:36286) 2023-03-27T10:01:55.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.911+0000 7f3cebfff700 1 -- 172.21.15.112:0/1050672059 learned_addr learned my addr 172.21.15.112:0/1050672059 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-03-27T10:01:55.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.911+0000 7f3cf0d56700 1 -- 172.21.15.112:0/1050672059 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3cec0b07f0 msgr2=0x7f3cec071e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:01:55.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.911+0000 7f3cf0d56700 1 --2- 172.21.15.112:0/1050672059 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3cec0b07f0 0x7f3cec071e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:55.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.911+0000 7f3cf0d56700 1 -- 172.21.15.112:0/1050672059 >> v1:172.21.15.112:6789/0 conn(0x7f3cec0723d0 legacy=0x7f3cec158650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:55.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.911+0000 7f3cf0d56700 1 -- 172.21.15.112:0/1050672059 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cec0710e0 con 0x7f3cec14eff0 2023-03-27T10:01:55.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.911+0000 7f3cebfff700 1 --2- 172.21.15.112:0/1050672059 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3cec0b07f0 0x7f3cec071e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-27T10:01:55.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.911+0000 7f3cf0d56700 1 --2- 172.21.15.112:0/1050672059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cec14eff0 0x7f3cec155780 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0009840 tx=0x7f3ce0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7070b6d82e8749 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:01:55.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.911+0000 7f3ceb7fe700 1 -- 172.21.15.112:0/1050672059 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce0011830 con 0x7f3cec14eff0 2023-03-27T10:01:55.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.912+0000 7f3ceb7fe700 1 -- 172.21.15.112:0/1050672059 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ce0011990 con 0x7f3cec14eff0 2023-03-27T10:01:55.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.912+0000 7f3ceb7fe700 1 -- 172.21.15.112:0/1050672059 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce0011d20 con 0x7f3cec14eff0 2023-03-27T10:01:55.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.912+0000 7f3cf2fba700 1 -- 172.21.15.112:0/1050672059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cec14eff0 msgr2=0x7f3cec155780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:55.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.912+0000 7f3cf2fba700 1 --2- 172.21.15.112:0/1050672059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cec14eff0 0x7f3cec155780 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0009840 tx=0x7f3ce0009b50 comp rx=0 tx=0).stop 2023-03-27T10:01:55.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.912+0000 7f3cf2fba700 1 -- 172.21.15.112:0/1050672059 shutdown_connections 2023-03-27T10:01:55.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.912+0000 7f3cf2fba700 1 --2- 172.21.15.112:0/1050672059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cec14eff0 0x7f3cec155780 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:55.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.912+0000 7f3cf2fba700 1 --2- 172.21.15.112:0/1050672059 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3cec0b07f0 0x7f3cec071e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:55.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.912+0000 7f3cf2fba700 1 -- 172.21.15.112:0/1050672059 >> 172.21.15.112:0/1050672059 conn(0x7f3cec0c8720 msgr2=0x7f3cec06e690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:01:55.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.912+0000 7f3cf2fba700 1 -- 172.21.15.112:0/1050672059 shutdown_connections 2023-03-27T10:01:55.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.912+0000 7f3cf2fba700 1 -- 172.21.15.112:0/1050672059 wait complete. 2023-03-27T10:01:55.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.913+0000 7f3cf2fba700 1 Processor -- start 2023-03-27T10:01:55.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.913+0000 7f3cf2fba700 1 -- start start 2023-03-27T10:01:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.914+0000 7f3cf2fba700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cec0723d0 0x7f3cec1ee530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.914+0000 7f3cf2fba700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3cec0b07f0 0x7f3cec150bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.914+0000 7f3cf2fba700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f3cec14eff0 0x7f3cec142570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.914+0000 7f3cf2fba700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cec143cb0 con 0x7f3cec14eff0 2023-03-27T10:01:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.914+0000 7f3cf2fba700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cec1510f0 con 0x7f3cec0723d0 2023-03-27T10:01:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.914+0000 7f3cf2fba700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cec142ab0 con 0x7f3cec0b07f0 2023-03-27T10:01:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.914+0000 7f3cebfff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3cec0b07f0 0x7f3cec150bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.914+0000 7f3cf0d56700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cec0723d0 0x7f3cec1ee530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:55.925 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.914+0000 7f3cf0d56700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cec0723d0 0x7f3cec1ee530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36292/0 (socket says 172.21.15.112:36292) 2023-03-27T10:01:55.925 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.914+0000 7f3cf0d56700 1 -- 172.21.15.112:0/2471331305 learned_addr learned my addr 172.21.15.112:0/2471331305 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-03-27T10:01:55.925 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.915+0000 7f3cf1557700 1 --2- 172.21.15.112:0/2471331305 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f3cec14eff0 0x7f3cec142570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:55.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.915+0000 7f3cebfff700 1 -- 172.21.15.112:0/2471331305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cec0723d0 msgr2=0x7f3cec1ee530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:55.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.915+0000 7f3cebfff700 1 --2- 172.21.15.112:0/2471331305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cec0723d0 0x7f3cec1ee530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:55.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.915+0000 7f3cebfff700 1 -- 172.21.15.112:0/2471331305 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f3cec14eff0 msgr2=0x7f3cec142570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:55.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.915+0000 7f3cebfff700 1 --2- 172.21.15.112:0/2471331305 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f3cec14eff0 0x7f3cec142570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:55.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.915+0000 7f3cebfff700 1 -- 172.21.15.112:0/2471331305 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd800b7f0 con 0x7f3cec0b07f0 2023-03-27T10:01:55.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.915+0000 7f3cf0d56700 1 --2- 172.21.15.112:0/2471331305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cec0723d0 0x7f3cec1ee530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-27T10:01:55.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.915+0000 7f3cebfff700 1 --2- 172.21.15.112:0/2471331305 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3cec0b07f0 0x7f3cec150bb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800de10 tx=0x7f3cd80103a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:01:55.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.915+0000 7f3ce97fa700 1 -- 172.21.15.112:0/2471331305 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8012040 con 0x7f3cec0b07f0 2023-03-27T10:01:55.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.915+0000 7f3ce97fa700 1 -- 172.21.15.112:0/2471331305 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd8010dc0 con 0x7f3cec0b07f0 2023-03-27T10:01:55.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.915+0000 7f3ce97fa700 1 -- 172.21.15.112:0/2471331305 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8004270 con 0x7f3cec0b07f0 2023-03-27T10:01:55.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.915+0000 7f3cf2fba700 1 -- 172.21.15.112:0/2471331305 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce00094a0 con 0x7f3cec0b07f0 2023-03-27T10:01:55.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.915+0000 7f3cf2fba700 1 -- 172.21.15.112:0/2471331305 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cec0c5e10 con 0x7f3cec0b07f0 2023-03-27T10:01:55.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.915+0000 7f3ce97fa700 1 -- 172.21.15.112:0/2471331305 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f3cd8005a20 con 0x7f3cec0b07f0 2023-03-27T10:01:55.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.915+0000 7f3ce97fa700 1 -- 172.21.15.112:0/2471331305 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f3cd8003c40 con 0x7f3cec0b07f0 2023-03-27T10:01:55.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.916+0000 7f3cf2fba700 1 -- 172.21.15.112:0/2471331305 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cec14f3c0 con 0x7f3cec0b07f0 2023-03-27T10:01:55.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:55.918+0000 7f3ce97fa700 1 -- 172.21.15.112:0/2471331305 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72400 (secure 0 0 0) 0x7f3cd80036a0 con 0x7f3cec0b07f0 2023-03-27T10:01:56.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.002+0000 7f3cf2fba700 1 -- 172.21.15.112:0/2471331305 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd new", "uuid": "6c839571-b490-4e66-92ac-0d7301c2e677", "id": 5} v 0) v1 -- 0x7f3cec0b0ea0 con 0x7f3cec0b07f0 2023-03-27T10:01:56.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.008+0000 7f3ce97fa700 1 -- 172.21.15.112:0/2471331305 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6c839571-b490-4e66-92ac-0d7301c2e677", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3cd8003840 con 0x7f3cec0b07f0 2023-03-27T10:01:56.010 INFO:teuthology.orchestra.run.smithi112.stdout:5 2023-03-27T10:01:56.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.009+0000 7f3cf2fba700 1 -- 172.21.15.112:0/2471331305 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3cec0b07f0 msgr2=0x7f3cec150bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:56.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.009+0000 7f3cf2fba700 1 --2- 172.21.15.112:0/2471331305 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3cec0b07f0 0x7f3cec150bb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800de10 tx=0x7f3cd80103a0 comp rx=0 tx=0).stop 2023-03-27T10:01:56.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.009+0000 7f3cf2fba700 1 -- 172.21.15.112:0/2471331305 shutdown_connections 2023-03-27T10:01:56.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.009+0000 7f3cf2fba700 1 --2- 172.21.15.112:0/2471331305 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f3cec14eff0 0x7f3cec142570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:56.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.009+0000 7f3cf2fba700 1 --2- 172.21.15.112:0/2471331305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cec0723d0 0x7f3cec1ee530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:56.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.009+0000 7f3cf2fba700 1 --2- 172.21.15.112:0/2471331305 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3cec0b07f0 0x7f3cec150bb0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:56.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.009+0000 7f3cf2fba700 1 -- 172.21.15.112:0/2471331305 >> 172.21.15.112:0/2471331305 conn(0x7f3cec0c8720 msgr2=0x7f3cec14d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:01:56.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.009+0000 7f3cf2fba700 1 -- 172.21.15.112:0/2471331305 shutdown_connections 2023-03-27T10:01:56.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.009+0000 7f3cf2fba700 1 -- 172.21.15.112:0/2471331305 wait complete. 2023-03-27T10:01:56.017 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-03-27T10:01:56.015+0000 7f105b615200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-03-27T10:01:56.021 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new 871b1a2b-f0ed-4faf-8f9a-a5a0973e11c9 6 2023-03-27T10:01:56.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.146+0000 7fef26627700 1 Processor -- start 2023-03-27T10:01:56.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.146+0000 7fef26627700 1 -- start start 2023-03-27T10:01:56.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.146+0000 7fef26627700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7fef20152c40 0x7fef200c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:56.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.146+0000 7fef26627700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fef2015c940 0x7fef2006f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:56.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.146+0000 7fef26627700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef20159730 con 0x7fef2006f570 2023-03-27T10:01:56.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.146+0000 7fef26627700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef200c84c0 con 0x7fef20152c40 2023-03-27T10:01:56.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.146+0000 7fef26627700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef200c86b0 con 0x7fef2015c940 2023-03-27T10:01:56.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.146+0000 7fef1f7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fef2015c940 0x7fef2006f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:56.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.147+0000 7fef1f7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fef2015c940 0x7fef2006f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36300/0 (socket says 172.21.15.112:36300) 2023-03-27T10:01:56.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.147+0000 7fef1f7fe700 1 -- 172.21.15.112:0/3420119992 learned_addr learned my addr 172.21.15.112:0/3420119992 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-03-27T10:01:56.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.147+0000 7fef1ffff700 1 --2- 172.21.15.112:0/3420119992 >> v2:172.21.15.23:3300/0 conn(0x7fef20152c40 0x7fef200c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:56.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.147+0000 7fef1f7fe700 1 -- 172.21.15.112:0/3420119992 >> v2:172.21.15.23:3300/0 conn(0x7fef20152c40 msgr2=0x7fef200c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:01:56.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.147+0000 7fef1f7fe700 1 --2- 172.21.15.112:0/3420119992 >> v2:172.21.15.23:3300/0 conn(0x7fef20152c40 0x7fef200c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:56.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.147+0000 7fef1f7fe700 1 -- 172.21.15.112:0/3420119992 >> v1:172.21.15.112:6789/0 conn(0x7fef2006f570 legacy=0x7fef20071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:56.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.147+0000 7fef1f7fe700 1 -- 172.21.15.112:0/3420119992 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef20158b20 con 0x7fef2015c940 2023-03-27T10:01:56.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.147+0000 7fef1f7fe700 1 --2- 172.21.15.112:0/3420119992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef2015c940 0x7fef2006f030 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fef2006fae0 tx=0x7fef14009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dd5363e1a65c4f3 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:01:56.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.147+0000 7fef1effd700 1 -- 172.21.15.112:0/3420119992 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1400b870 con 0x7fef2015c940 2023-03-27T10:01:56.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.147+0000 7fef1effd700 1 -- 172.21.15.112:0/3420119992 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef1400b9d0 con 0x7fef2015c940 2023-03-27T10:01:56.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.147+0000 7fef1effd700 1 -- 172.21.15.112:0/3420119992 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1400bd60 con 0x7fef2015c940 2023-03-27T10:01:56.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.148+0000 7fef26627700 1 -- 172.21.15.112:0/3420119992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef2015c940 msgr2=0x7fef2006f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:56.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.148+0000 7fef26627700 1 --2- 172.21.15.112:0/3420119992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef2015c940 0x7fef2006f030 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fef2006fae0 tx=0x7fef14009bd0 comp rx=0 tx=0).stop 2023-03-27T10:01:56.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.148+0000 7fef26627700 1 -- 172.21.15.112:0/3420119992 shutdown_connections 2023-03-27T10:01:56.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.148+0000 7fef26627700 1 --2- 172.21.15.112:0/3420119992 >> v2:172.21.15.23:3300/0 conn(0x7fef20152c40 0x7fef200c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:56.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.148+0000 7fef26627700 1 --2- 172.21.15.112:0/3420119992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef2015c940 0x7fef2006f030 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:56.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.148+0000 7fef26627700 1 -- 172.21.15.112:0/3420119992 >> 172.21.15.112:0/3420119992 conn(0x7fef200660c0 msgr2=0x7fef2006d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:01:56.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.148+0000 7fef26627700 1 -- 172.21.15.112:0/3420119992 shutdown_connections 2023-03-27T10:01:56.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.148+0000 7fef26627700 1 -- 172.21.15.112:0/3420119992 wait complete. 2023-03-27T10:01:56.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.149+0000 7fef26627700 1 Processor -- start 2023-03-27T10:01:56.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.149+0000 7fef26627700 1 -- start start 2023-03-27T10:01:56.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.150+0000 7fef26627700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fef2006f570 0x7fef20159b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:56.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.150+0000 7fef26627700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fef20152c40 0x7fef2015bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:56.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.150+0000 7fef26627700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef2015c940 0x7fef2014b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:56.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.150+0000 7fef26627700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2015aad0 con 0x7fef2006f570 2023-03-27T10:01:56.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.150+0000 7fef26627700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef200cb570 con 0x7fef2015c940 2023-03-27T10:01:56.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.150+0000 7fef26627700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef200c5530 con 0x7fef20152c40 2023-03-27T10:01:56.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.150+0000 7fef24bc4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef2015c940 0x7fef2014b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:56.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.150+0000 7fef24bc4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef2015c940 0x7fef2014b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36306/0 (socket says 172.21.15.112:36306) 2023-03-27T10:01:56.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.150+0000 7fef1f7fe700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fef20152c40 0x7fef2015bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:56.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.150+0000 7fef24bc4700 1 -- 172.21.15.112:0/4272684622 learned_addr learned my addr 172.21.15.112:0/4272684622 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-03-27T10:01:56.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.150+0000 7fef1ffff700 1 --2- 172.21.15.112:0/4272684622 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fef2006f570 0x7fef20159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:56.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.151+0000 7fef24bc4700 1 -- 172.21.15.112:0/4272684622 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fef20152c40 msgr2=0x7fef2015bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:56.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.151+0000 7fef24bc4700 1 --2- 172.21.15.112:0/4272684622 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fef20152c40 0x7fef2015bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:56.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.151+0000 7fef24bc4700 1 -- 172.21.15.112:0/4272684622 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fef2006f570 msgr2=0x7fef20159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:56.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.151+0000 7fef24bc4700 1 --2- 172.21.15.112:0/4272684622 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fef2006f570 0x7fef20159b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:56.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.151+0000 7fef24bc4700 1 -- 172.21.15.112:0/4272684622 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef14009550 con 0x7fef2015c940 2023-03-27T10:01:56.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.151+0000 7fef1f7fe700 1 --2- 172.21.15.112:0/4272684622 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fef20152c40 0x7fef2015bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-27T10:01:56.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.151+0000 7fef24bc4700 1 --2- 172.21.15.112:0/4272684622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef2015c940 0x7fef2014b650 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fef1800cc80 tx=0x7fef1800cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:01:56.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.151+0000 7fef1cff9700 1 -- 172.21.15.112:0/4272684622 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef18003c00 con 0x7fef2015c940 2023-03-27T10:01:56.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.151+0000 7fef1cff9700 1 -- 172.21.15.112:0/4272684622 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef18003d60 con 0x7fef2015c940 2023-03-27T10:01:56.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.151+0000 7fef26627700 1 -- 172.21.15.112:0/4272684622 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef201590e0 con 0x7fef2015c940 2023-03-27T10:01:56.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.151+0000 7fef1cff9700 1 -- 172.21.15.112:0/4272684622 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1800aaa0 con 0x7fef2015c940 2023-03-27T10:01:56.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.151+0000 7fef26627700 1 -- 172.21.15.112:0/4272684622 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef20158e00 con 0x7fef2015c940 2023-03-27T10:01:56.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.151+0000 7fef1cff9700 1 -- 172.21.15.112:0/4272684622 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fef18010050 con 0x7fef2015c940 2023-03-27T10:01:56.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.151+0000 7fef1cff9700 1 -- 172.21.15.112:0/4272684622 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fef18012a10 con 0x7fef2015c940 2023-03-27T10:01:56.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.152+0000 7fef26627700 1 -- 172.21.15.112:0/4272684622 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef201593c0 con 0x7fef2015c940 2023-03-27T10:01:56.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.154+0000 7fef1cff9700 1 -- 172.21.15.112:0/4272684622 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72400 (secure 0 0 0) 0x7fef18012c70 con 0x7fef2015c940 2023-03-27T10:01:56.178 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-03-27T10:01:56.178+0000 7f105b615200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-03-27T10:01:56.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.238+0000 7fef26627700 1 -- 172.21.15.112:0/4272684622 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd new", "uuid": "871b1a2b-f0ed-4faf-8f9a-a5a0973e11c9", "id": 6} v 0) v1 -- 0x7fef20069220 con 0x7fef2015c940 2023-03-27T10:01:56.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.247+0000 7fef1cff9700 1 -- 172.21.15.112:0/4272684622 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "871b1a2b-f0ed-4faf-8f9a-a5a0973e11c9", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fef18012e10 con 0x7fef2015c940 2023-03-27T10:01:56.248 INFO:teuthology.orchestra.run.smithi112.stdout:6 2023-03-27T10:01:56.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.248+0000 7fef26627700 1 -- 172.21.15.112:0/4272684622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef2015c940 msgr2=0x7fef2014b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:56.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.248+0000 7fef26627700 1 --2- 172.21.15.112:0/4272684622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef2015c940 0x7fef2014b650 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fef1800cc80 tx=0x7fef1800cf90 comp rx=0 tx=0).stop 2023-03-27T10:01:56.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.248+0000 7fef26627700 1 -- 172.21.15.112:0/4272684622 shutdown_connections 2023-03-27T10:01:56.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.248+0000 7fef26627700 1 --2- 172.21.15.112:0/4272684622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef2015c940 0x7fef2014b650 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:56.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.248+0000 7fef26627700 1 --2- 172.21.15.112:0/4272684622 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fef2006f570 0x7fef20159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:56.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.248+0000 7fef26627700 1 --2- 172.21.15.112:0/4272684622 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fef20152c40 0x7fef2015bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:56.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.249+0000 7fef26627700 1 -- 172.21.15.112:0/4272684622 >> 172.21.15.112:0/4272684622 conn(0x7fef200660c0 msgr2=0x7fef2006b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:01:56.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.249+0000 7fef26627700 1 -- 172.21.15.112:0/4272684622 shutdown_connections 2023-03-27T10:01:56.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.249+0000 7fef26627700 1 -- 172.21.15.112:0/4272684622 wait complete. 2023-03-27T10:01:56.260 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new d8228353-34b5-4363-84d8-642512f5bc06 7 2023-03-27T10:01:56.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.380+0000 7ffb10fe4700 1 Processor -- start 2023-03-27T10:01:56.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.380+0000 7ffb10fe4700 1 -- start start 2023-03-27T10:01:56.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.381+0000 7ffb10fe4700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb0c0c8810 0x7ffb0c0cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:56.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.381+0000 7ffb10fe4700 1 -- --> v1:172.21.15.23:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb0c0c3220 con 0x7ffb0c0c77f0 2023-03-27T10:01:56.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.381+0000 7ffb10fe4700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb0c070a20 con 0x7ffb0c1508f0 2023-03-27T10:01:56.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.381+0000 7ffb10fe4700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c0c1cb0 con 0x7ffb0c0c8810 2023-03-27T10:01:56.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.381+0000 7ffb0a59c700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7ffb0c1508f0 0x7ffb0c0c2140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:36260/0 (socket says 172.21.15.112:36260) 2023-03-27T10:01:56.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.381+0000 7ffb0a59c700 1 -- 172.21.15.112:0/2367945091 learned_addr learned my addr 172.21.15.112:0/2367945091 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-03-27T10:01:56.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.381+0000 7ffb09d9b700 1 --2- 172.21.15.112:0/2367945091 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb0c0c8810 0x7ffb0c0cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:56.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.382+0000 7ffb0959a700 1 -- 172.21.15.112:0/2367945091 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2734157954 0 0) 0x7ffb0c070a20 con 0x7ffb0c1508f0 2023-03-27T10:01:56.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.382+0000 7ffb0959a700 1 -- 172.21.15.112:0/2367945091 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffaf80035b0 con 0x7ffb0c1508f0 2023-03-27T10:01:56.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.382+0000 7ffb09d9b700 1 -- 172.21.15.112:0/2367945091 >> v1:172.21.15.112:6789/0 conn(0x7ffb0c1508f0 legacy=0x7ffb0c0c2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:56.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.382+0000 7ffb09d9b700 1 -- 172.21.15.112:0/2367945091 >> v1:172.21.15.23:6789/0 conn(0x7ffb0c0c77f0 legacy=0x7ffb0c161050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:56.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.382+0000 7ffb09d9b700 1 -- 172.21.15.112:0/2367945091 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb0c157570 con 0x7ffb0c0c8810 2023-03-27T10:01:56.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.382+0000 7ffb09d9b700 1 --2- 172.21.15.112:0/2367945091 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb0c0c8810 0x7ffb0c0cc8b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4009870 tx=0x7ffaf4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e36db8a0e351f3f server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:01:56.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.382+0000 7ffb0959a700 1 -- 172.21.15.112:0/2367945091 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf4011830 con 0x7ffb0c0c8810 2023-03-27T10:01:56.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.382+0000 7ffb0959a700 1 -- 172.21.15.112:0/2367945091 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf4011990 con 0x7ffb0c0c8810 2023-03-27T10:01:56.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.382+0000 7ffb0959a700 1 -- 172.21.15.112:0/2367945091 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf4011d20 con 0x7ffb0c0c8810 2023-03-27T10:01:56.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.383+0000 7ffb10fe4700 1 -- 172.21.15.112:0/2367945091 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb0c0c8810 msgr2=0x7ffb0c0cc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:56.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.383+0000 7ffb10fe4700 1 --2- 172.21.15.112:0/2367945091 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb0c0c8810 0x7ffb0c0cc8b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4009870 tx=0x7ffaf4009b80 comp rx=0 tx=0).stop 2023-03-27T10:01:56.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.383+0000 7ffb10fe4700 1 -- 172.21.15.112:0/2367945091 shutdown_connections 2023-03-27T10:01:56.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.383+0000 7ffb10fe4700 1 --2- 172.21.15.112:0/2367945091 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb0c0c8810 0x7ffb0c0cc8b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:56.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.383+0000 7ffb10fe4700 1 -- 172.21.15.112:0/2367945091 >> 172.21.15.112:0/2367945091 conn(0x7ffb0c06a1f0 msgr2=0x7ffb0c06a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:01:56.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.383+0000 7ffb10fe4700 1 -- 172.21.15.112:0/2367945091 shutdown_connections 2023-03-27T10:01:56.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.383+0000 7ffb10fe4700 1 -- 172.21.15.112:0/2367945091 wait complete. 2023-03-27T10:01:56.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.384+0000 7ffb10fe4700 1 Processor -- start 2023-03-27T10:01:56.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.385+0000 7ffb10fe4700 1 -- start start 2023-03-27T10:01:56.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.385+0000 7ffb10fe4700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7ffb0c0c77f0 0x7ffb0c147f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:56.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.385+0000 7ffb10fe4700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb0c0c8810 0x7ffb0c1484d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:56.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.385+0000 7ffb10fe4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb0c1508f0 0x7ffb0c0c4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:56.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.385+0000 7ffb10fe4700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c1591a0 con 0x7ffb0c0c77f0 2023-03-27T10:01:56.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.385+0000 7ffb10fe4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c1efb80 con 0x7ffb0c1508f0 2023-03-27T10:01:56.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.385+0000 7ffb10fe4700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c15a740 con 0x7ffb0c0c8810 2023-03-27T10:01:56.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.385+0000 7ffb09d9b700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb0c0c8810 0x7ffb0c1484d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:56.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.385+0000 7ffb09d9b700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb0c0c8810 0x7ffb0c1484d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.112:54352/0 (socket says 172.21.15.112:54352) 2023-03-27T10:01:56.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.385+0000 7ffb09d9b700 1 -- 172.21.15.112:0/3028474172 learned_addr learned my addr 172.21.15.112:0/3028474172 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-03-27T10:01:56.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.385+0000 7ffb0ad9d700 1 --2- 172.21.15.112:0/3028474172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb0c1508f0 0x7ffb0c0c4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:56.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.385+0000 7ffb0a59c700 1 --2- 172.21.15.112:0/3028474172 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7ffb0c0c77f0 0x7ffb0c147f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:56.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.386+0000 7ffb09d9b700 1 -- 172.21.15.112:0/3028474172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb0c1508f0 msgr2=0x7ffb0c0c4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:56.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.386+0000 7ffb09d9b700 1 --2- 172.21.15.112:0/3028474172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb0c1508f0 0x7ffb0c0c4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:56.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.386+0000 7ffb09d9b700 1 -- 172.21.15.112:0/3028474172 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7ffb0c0c77f0 msgr2=0x7ffb0c147f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:56.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.386+0000 7ffb09d9b700 1 --2- 172.21.15.112:0/3028474172 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7ffb0c0c77f0 0x7ffb0c147f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:56.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.386+0000 7ffb09d9b700 1 -- 172.21.15.112:0/3028474172 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf40094a0 con 0x7ffb0c0c8810 2023-03-27T10:01:56.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.386+0000 7ffb09d9b700 1 --2- 172.21.15.112:0/3028474172 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb0c0c8810 0x7ffb0c1484d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ffaf40090b0 tx=0x7ffaf4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:01:56.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.386+0000 7ffb02ffd700 1 -- 172.21.15.112:0/3028474172 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf40036a0 con 0x7ffb0c0c8810 2023-03-27T10:01:56.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.386+0000 7ffb02ffd700 1 -- 172.21.15.112:0/3028474172 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf401f070 con 0x7ffb0c0c8810 2023-03-27T10:01:56.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.386+0000 7ffb10fe4700 1 -- 172.21.15.112:0/3028474172 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb0c0c1940 con 0x7ffb0c0c8810 2023-03-27T10:01:56.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.386+0000 7ffb02ffd700 1 -- 172.21.15.112:0/3028474172 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf4003a60 con 0x7ffb0c0c8810 2023-03-27T10:01:56.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.386+0000 7ffb10fe4700 1 -- 172.21.15.112:0/3028474172 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb0c070c70 con 0x7ffb0c0c8810 2023-03-27T10:01:56.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.386+0000 7ffb02ffd700 1 -- 172.21.15.112:0/3028474172 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7ffaf400f050 con 0x7ffb0c0c8810 2023-03-27T10:01:56.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.386+0000 7ffb02ffd700 1 -- 172.21.15.112:0/3028474172 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7ffaf4021070 con 0x7ffb0c0c8810 2023-03-27T10:01:56.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.387+0000 7ffb10fe4700 1 -- 172.21.15.112:0/3028474172 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb0c0c2850 con 0x7ffb0c0c8810 2023-03-27T10:01:56.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.389+0000 7ffb02ffd700 1 -- 172.21.15.112:0/3028474172 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72400 (secure 0 0 0) 0x7ffaf4015020 con 0x7ffb0c0c8810 2023-03-27T10:01:56.451 INFO:tasks.ceph.mgr.y.smithi023.stderr:2023-03-27T10:01:56.452+0000 7f6256375200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-03-27T10:01:56.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.472+0000 7ffb10fe4700 1 -- 172.21.15.112:0/3028474172 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d8228353-34b5-4363-84d8-642512f5bc06", "id": 7} v 0) v1 -- 0x7ffb0c1efd30 con 0x7ffb0c0c8810 2023-03-27T10:01:56.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.480+0000 7ffb02ffd700 1 -- 172.21.15.112:0/3028474172 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d8228353-34b5-4363-84d8-642512f5bc06", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7ffaf4003c90 con 0x7ffb0c0c8810 2023-03-27T10:01:56.481 INFO:teuthology.orchestra.run.smithi112.stdout:7 2023-03-27T10:01:56.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.481+0000 7ffb10fe4700 1 -- 172.21.15.112:0/3028474172 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb0c0c8810 msgr2=0x7ffb0c1484d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:56.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.481+0000 7ffb10fe4700 1 --2- 172.21.15.112:0/3028474172 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb0c0c8810 0x7ffb0c1484d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ffaf40090b0 tx=0x7ffaf4004750 comp rx=0 tx=0).stop 2023-03-27T10:01:56.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.481+0000 7ffb10fe4700 1 -- 172.21.15.112:0/3028474172 shutdown_connections 2023-03-27T10:01:56.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.481+0000 7ffb10fe4700 1 --2- 172.21.15.112:0/3028474172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb0c1508f0 0x7ffb0c0c4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:56.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.481+0000 7ffb10fe4700 1 --2- 172.21.15.112:0/3028474172 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7ffb0c0c77f0 0x7ffb0c147f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:56.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.481+0000 7ffb10fe4700 1 --2- 172.21.15.112:0/3028474172 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb0c0c8810 0x7ffb0c1484d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:56.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.481+0000 7ffb10fe4700 1 -- 172.21.15.112:0/3028474172 >> 172.21.15.112:0/3028474172 conn(0x7ffb0c06a1f0 msgr2=0x7ffb0c0b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:01:56.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.481+0000 7ffb10fe4700 1 -- 172.21.15.112:0/3028474172 shutdown_connections 2023-03-27T10:01:56.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-03-27T10:01:56.481+0000 7ffb10fe4700 1 -- 172.21.15.112:0/3028474172 wait complete. 2023-03-27T10:01:56.493 INFO:tasks.ceph.osd.0:Restarting daemon 2023-03-27T10:01:56.493 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-03-27T10:01:56.497 INFO:tasks.ceph.osd.0:Started 2023-03-27T10:01:56.498 INFO:tasks.ceph.osd.1:Restarting daemon 2023-03-27T10:01:56.498 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-03-27T10:01:56.504 INFO:tasks.ceph.osd.1:Started 2023-03-27T10:01:56.505 INFO:tasks.ceph.osd.2:Restarting daemon 2023-03-27T10:01:56.505 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-03-27T10:01:56.510 INFO:tasks.ceph.osd.2:Started 2023-03-27T10:01:56.510 INFO:tasks.ceph.osd.3:Restarting daemon 2023-03-27T10:01:56.511 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-03-27T10:01:56.516 INFO:tasks.ceph.osd.3:Started 2023-03-27T10:01:56.516 INFO:tasks.ceph.osd.4:Restarting daemon 2023-03-27T10:01:56.516 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-03-27T10:01:56.519 INFO:tasks.ceph.osd.4:Started 2023-03-27T10:01:56.519 INFO:tasks.ceph.osd.5:Restarting daemon 2023-03-27T10:01:56.519 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-03-27T10:01:56.521 INFO:tasks.ceph.osd.5:Started 2023-03-27T10:01:56.522 INFO:tasks.ceph.osd.6:Restarting daemon 2023-03-27T10:01:56.522 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-03-27T10:01:56.524 INFO:tasks.ceph.osd.6:Started 2023-03-27T10:01:56.524 INFO:tasks.ceph.osd.7:Restarting daemon 2023-03-27T10:01:56.524 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-03-27T10:01:56.526 INFO:tasks.ceph.osd.7:Started 2023-03-27T10:01:56.527 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-27T10:01:56.677 INFO:tasks.ceph.mgr.y.smithi023.stderr:2023-03-27T10:01:56.677+0000 7f6256375200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-03-27T10:01:56.795 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.795+0000 7f2f5a5ff700 1 Processor -- start 2023-03-27T10:01:56.795 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.796+0000 7f2f5a5ff700 1 -- start start 2023-03-27T10:01:56.796 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.797+0000 7f2f5a5ff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2f540c9520 0x7f2f540c8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:56.796 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.797+0000 7f2f5a5ff700 1 -- --> v1:172.21.15.23:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f540ca730 con 0x7f2f5406b440 2023-03-27T10:01:56.796 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.797+0000 7f2f5a5ff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f540cf9f0 con 0x7f2f540c6990 2023-03-27T10:01:56.796 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.797+0000 7f2f5a5ff700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f54158a00 con 0x7f2f540c9520 2023-03-27T10:01:56.798 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.797+0000 7f2f537fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2f540c9520 0x7f2f540c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:56.798 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.797+0000 7f2f58b9c700 1 --1- >> v1:172.21.15.23:6789/0 conn(0x7f2f5406b440 0x7f2f5406b840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.23:6789/0 says I am v1:172.21.15.23:34676/0 (socket says 172.21.15.23:34676) 2023-03-27T10:01:56.798 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.797+0000 7f2f58b9c700 1 -- 172.21.15.23:0/963352962 learned_addr learned my addr 172.21.15.23:0/963352962 (peer_addr_for_me v1:172.21.15.23:0/0) 2023-03-27T10:01:56.798 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.797+0000 7f2f53fff700 1 --1- 172.21.15.23:0/963352962 >> v1:172.21.15.112:6789/0 conn(0x7f2f540c6990 0x7f2f540c6d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.23:58892/0 (socket says 172.21.15.23:58892) 2023-03-27T10:01:56.799 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.798+0000 7f2f537fe700 1 -- 172.21.15.23:0/963352962 >> v1:172.21.15.112:6789/0 conn(0x7f2f540c6990 legacy=0x7f2f540c6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:56.799 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.798+0000 7f2f537fe700 1 -- 172.21.15.23:0/963352962 >> v1:172.21.15.23:6789/0 conn(0x7f2f5406b440 legacy=0x7f2f5406b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:56.799 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.798+0000 7f2f52ffd700 1 -- 172.21.15.23:0/963352962 <== mon.0 v1:172.21.15.23:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 832381672 0 0) 0x7f2f540ca730 con 0x7f2f5406b440 2023-03-27T10:01:56.799 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.798+0000 7f2f537fe700 1 -- 172.21.15.23:0/963352962 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f54158ce0 con 0x7f2f540c9520 2023-03-27T10:01:56.799 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.798+0000 7f2f537fe700 1 --2- 172.21.15.23:0/963352962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f540c9520 0x7f2f540c8820 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2f480097c0 tx=0x7f2f48009ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3811765224992829 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:01:56.799 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.798+0000 7f2f52ffd700 1 -- 172.21.15.23:0/963352962 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f48004230 con 0x7f2f540c9520 2023-03-27T10:01:56.799 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.798+0000 7f2f52ffd700 1 -- 172.21.15.23:0/963352962 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f48004390 con 0x7f2f540c9520 2023-03-27T10:01:56.800 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.798+0000 7f2f52ffd700 1 -- 172.21.15.23:0/963352962 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f480036a0 con 0x7f2f540c9520 2023-03-27T10:01:56.800 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.799+0000 7f2f5a5ff700 1 -- 172.21.15.23:0/963352962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f540c9520 msgr2=0x7f2f540c8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:56.800 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.799+0000 7f2f5a5ff700 1 --2- 172.21.15.23:0/963352962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f540c9520 0x7f2f540c8820 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2f480097c0 tx=0x7f2f48009ad0 comp rx=0 tx=0).stop 2023-03-27T10:01:56.800 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.799+0000 7f2f5a5ff700 1 -- 172.21.15.23:0/963352962 shutdown_connections 2023-03-27T10:01:56.800 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.799+0000 7f2f5a5ff700 1 --2- 172.21.15.23:0/963352962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f540c9520 0x7f2f540c8820 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:56.800 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.799+0000 7f2f5a5ff700 1 -- 172.21.15.23:0/963352962 >> 172.21.15.23:0/963352962 conn(0x7f2f54071920 msgr2=0x7f2f54071cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:01:56.800 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.799+0000 7f2f5a5ff700 1 -- 172.21.15.23:0/963352962 shutdown_connections 2023-03-27T10:01:56.800 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.799+0000 7f2f5a5ff700 1 -- 172.21.15.23:0/963352962 wait complete. 2023-03-27T10:01:56.801 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.801+0000 7f2f5a5ff700 1 Processor -- start 2023-03-27T10:01:56.801 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.801+0000 7f2f5a5ff700 1 -- start start 2023-03-27T10:01:56.804 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.803+0000 7f2f5a5ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f5406b440 0x7f2f541599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:56.804 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.803+0000 7f2f5a5ff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f2f540c6990 0x7f2f541eec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:56.804 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.803+0000 7f2f5a5ff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f540c9520 0x7f2f540cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:56.804 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.803+0000 7f2f5a5ff700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f541ee770 con 0x7f2f540c6990 2023-03-27T10:01:56.804 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.803+0000 7f2f5a5ff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5415a5f0 con 0x7f2f540c9520 2023-03-27T10:01:56.804 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.803+0000 7f2f5a5ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5415cab0 con 0x7f2f5406b440 2023-03-27T10:01:56.805 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.803+0000 7f2f53fff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f5406b440 0x7f2f541599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:56.805 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.803+0000 7f2f58b9c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f540c9520 0x7f2f540cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:56.805 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.803+0000 7f2f537fe700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f2f540c6990 0x7f2f541eec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:56.805 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.803+0000 7f2f53fff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f5406b440 0x7f2f541599c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.23:39900/0 (socket says 172.21.15.23:39900) 2023-03-27T10:01:56.805 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.803+0000 7f2f53fff700 1 -- 172.21.15.23:0/3663430302 learned_addr learned my addr 172.21.15.23:0/3663430302 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:01:56.805 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.803+0000 7f2f53fff700 1 -- 172.21.15.23:0/3663430302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f540c9520 msgr2=0x7f2f540cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:56.806 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.803+0000 7f2f53fff700 1 --2- 172.21.15.23:0/3663430302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f540c9520 0x7f2f540cabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:56.806 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.803+0000 7f2f53fff700 1 -- 172.21.15.23:0/3663430302 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f2f540c6990 msgr2=0x7f2f541eec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:56.806 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.803+0000 7f2f53fff700 1 --2- 172.21.15.23:0/3663430302 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f2f540c6990 0x7f2f541eec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:56.806 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.803+0000 7f2f53fff700 1 -- 172.21.15.23:0/3663430302 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f480094a0 con 0x7f2f5406b440 2023-03-27T10:01:56.807 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.803+0000 7f2f58b9c700 1 --2- 172.21.15.23:0/3663430302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f540c9520 0x7f2f540cabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-27T10:01:56.807 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.803+0000 7f2f537fe700 1 --2- 172.21.15.23:0/3663430302 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f2f540c6990 0x7f2f541eec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-27T10:01:56.807 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.804+0000 7f2f53fff700 1 --2- 172.21.15.23:0/3663430302 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f5406b440 0x7f2f541599c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2f4400c780 tx=0x7f2f4400ca90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:01:56.807 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.804+0000 7f2f50ff9700 1 -- 172.21.15.23:0/3663430302 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f44004c80 con 0x7f2f5406b440 2023-03-27T10:01:56.807 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.804+0000 7f2f50ff9700 1 -- 172.21.15.23:0/3663430302 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f44004de0 con 0x7f2f5406b440 2023-03-27T10:01:56.807 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.804+0000 7f2f50ff9700 1 -- 172.21.15.23:0/3663430302 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f44007310 con 0x7f2f5406b440 2023-03-27T10:01:56.807 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.804+0000 7f2f5a5ff700 1 -- 172.21.15.23:0/3663430302 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f541592a0 con 0x7f2f5406b440 2023-03-27T10:01:56.807 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.804+0000 7f2f5a5ff700 1 -- 172.21.15.23:0/3663430302 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f54158fc0 con 0x7f2f5406b440 2023-03-27T10:01:56.808 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.804+0000 7f2f50ff9700 1 -- 172.21.15.23:0/3663430302 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f2f4401a020 con 0x7f2f5406b440 2023-03-27T10:01:56.808 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.804+0000 7f2f50ff9700 1 -- 172.21.15.23:0/3663430302 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f2f44013a90 con 0x7f2f5406b440 2023-03-27T10:01:56.808 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.805+0000 7f2f5a5ff700 1 -- 172.21.15.23:0/3663430302 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f54159580 con 0x7f2f5406b440 2023-03-27T10:01:56.808 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.807+0000 7f2f50ff9700 1 -- 172.21.15.23:0/3663430302 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72400 (secure 0 0 0) 0x7f2f4401d4a0 con 0x7f2f5406b440 2023-03-27T10:01:56.883 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.883+0000 7f2f5a5ff700 1 -- 172.21.15.23:0/3663430302 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f541eea50 con 0x7f2f5406b440 2023-03-27T10:01:56.884 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.884+0000 7f2f50ff9700 1 -- 172.21.15.23:0/3663430302 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f2f44013d60 con 0x7f2f5406b440 2023-03-27T10:01:56.884 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:01:56.884 INFO:teuthology.orchestra.run.smithi023.stdout:{"epoch":10,"fsid":"1d8d6e6e-0d21-4f51-91ee-d9434d4fb750","created":"2023-03-27T10:01:53.805778+0000","modified":"2023-03-27T10:01:56.475091+0000","last_up_change":"0.000000","last_in_change":"2023-03-27T10:01:56.475091+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"df086052-536b-4fb1-bcce-7aaf5a13911e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"03138001-a3fd-43fc-9797-a47643d9a3a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"964dd926-1160-44ef-aa34-b18965dc05f3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4ae5500f-cb24-44a8-a536-5c399c09e566","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"de228ab2-92b4-40bc-8dbd-8f2f4895dc22","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6c839571-b490-4e66-92ac-0d7301c2e677","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"871b1a2b-f0ed-4faf-8f9a-a5a0973e11c9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d8228353-34b5-4363-84d8-642512f5bc06","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-27T10:01:56.885 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.885+0000 7f2f5a5ff700 1 -- 172.21.15.23:0/3663430302 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f5406b440 msgr2=0x7f2f541599c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:56.885 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.885+0000 7f2f5a5ff700 1 --2- 172.21.15.23:0/3663430302 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f5406b440 0x7f2f541599c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2f4400c780 tx=0x7f2f4400ca90 comp rx=0 tx=0).stop 2023-03-27T10:01:56.886 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.885+0000 7f2f5a5ff700 1 -- 172.21.15.23:0/3663430302 shutdown_connections 2023-03-27T10:01:56.886 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.885+0000 7f2f5a5ff700 1 --2- 172.21.15.23:0/3663430302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f540c9520 0x7f2f540cabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:56.886 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.885+0000 7f2f5a5ff700 1 --2- 172.21.15.23:0/3663430302 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f5406b440 0x7f2f541599c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:56.886 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.885+0000 7f2f5a5ff700 1 --2- 172.21.15.23:0/3663430302 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f2f540c6990 0x7f2f541eec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:56.886 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.885+0000 7f2f5a5ff700 1 -- 172.21.15.23:0/3663430302 >> 172.21.15.23:0/3663430302 conn(0x7f2f54071920 msgr2=0x7f2f5406b810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:01:56.887 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.885+0000 7f2f5a5ff700 1 -- 172.21.15.23:0/3663430302 shutdown_connections 2023-03-27T10:01:56.887 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:01:56.885+0000 7f2f5a5ff700 1 -- 172.21.15.23:0/3663430302 wait complete. 2023-03-27T10:01:56.902 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-03-27T10:01:56.902 INFO:tasks.ceph:Waiting for OSDs to come up 2023-03-27T10:01:56.910 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-03-27T10:01:56.910+0000 7f105b615200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-03-27T10:01:56.984 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-03-27T10:01:56.984+0000 7f105b615200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-03-27T10:01:57.122 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-03-27T10:01:57.120+0000 7f105b615200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-03-27T10:01:57.130 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-03-27T10:01:57.130+0000 7f5f82a08580 -1 Falling back to public interface 2023-03-27T10:01:57.132 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-03-27T10:01:57.132+0000 7fa44f917580 -1 Falling back to public interface 2023-03-27T10:01:57.133 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-03-27T10:01:57.133+0000 7f6692b4d580 -1 Falling back to public interface 2023-03-27T10:01:57.145 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-03-27T10:01:57.146+0000 7fd9215fc580 -1 Falling back to public interface 2023-03-27T10:01:57.229 INFO:tasks.ceph.osd.0.smithi023.stderr:2023-03-27T10:01:57.229+0000 7efdea2e7580 -1 Falling back to public interface 2023-03-27T10:01:57.232 INFO:tasks.ceph.osd.1.smithi023.stderr:2023-03-27T10:01:57.233+0000 7efc17a6f580 -1 Falling back to public interface 2023-03-27T10:01:57.249 INFO:tasks.ceph.osd.3.smithi023.stderr:2023-03-27T10:01:57.250+0000 7f58f27bc580 -1 Falling back to public interface 2023-03-27T10:01:57.250 INFO:tasks.ceph.osd.2.smithi023.stderr:2023-03-27T10:01:57.251+0000 7f716e8d9580 -1 Falling back to public interface 2023-03-27T10:01:57.284 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-03-27T10:01:57.285+0000 7f105b615200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-03-27T10:01:57.416 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-03-27T10:01:57.416+0000 7f105b615200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-03-27T10:01:57.555 INFO:tasks.ceph.mgr.y.smithi023.stderr:2023-03-27T10:01:57.555+0000 7f6256375200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-03-27T10:01:57.652 INFO:tasks.ceph.mgr.y.smithi023.stderr:2023-03-27T10:01:57.653+0000 7f6256375200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-03-27T10:01:57.708 DEBUG:teuthology.orchestra.run.smithi023:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-03-27T10:01:57.763 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-03-27T10:01:57.763+0000 7f105b615200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-03-27T10:01:57.829 INFO:tasks.ceph.mgr.y.smithi023.stderr:2023-03-27T10:01:57.829+0000 7f6256375200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-03-27T10:01:57.877 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.877+0000 7f8f43dff700 1 Processor -- start 2023-03-27T10:01:57.877 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.877+0000 7f8f43dff700 1 -- start start 2023-03-27T10:01:57.878 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.879+0000 7f8f43dff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f3c0c4eb0 0x7f8f3c15c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:57.878 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.879+0000 7f8f43dff700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f8f3c15ce00 0x7f8f3c0cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:57.878 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.879+0000 7f8f43dff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f3c15c150 con 0x7f8f3c152e80 2023-03-27T10:01:57.879 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.879+0000 7f8f43dff700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f3c0ca100 con 0x7f8f3c15ce00 2023-03-27T10:01:57.879 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.879+0000 7f8f43dff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c1590a0 con 0x7f8f3c0c4eb0 2023-03-27T10:01:57.879 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.879+0000 7f8f41b9b700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f8f3c152e80 0x7f8f3c0c94d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.23:50086/0 (socket says 172.21.15.23:50086) 2023-03-27T10:01:57.879 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.879+0000 7f8f41b9b700 1 -- 172.21.15.23:0/3831956625 learned_addr learned my addr 172.21.15.23:0/3831956625 (peer_addr_for_me v1:172.21.15.23:0/0) 2023-03-27T10:01:57.879 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.879+0000 7f8f4139a700 1 --2- 172.21.15.23:0/3831956625 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f3c0c4eb0 0x7f8f3c15c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:57.880 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.879+0000 7f8f4239c700 1 --2- 172.21.15.23:0/3831956625 >> v2:172.21.15.23:3300/0 conn(0x7f8f3c15ce00 0x7f8f3c0cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:57.881 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.879+0000 7f8f40b99700 1 -- 172.21.15.23:0/3831956625 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3542295974 0 0) 0x7f8f3c15c150 con 0x7f8f3c152e80 2023-03-27T10:01:57.881 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.879+0000 7f8f40b99700 1 -- 172.21.15.23:0/3831956625 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f280035b0 con 0x7f8f3c152e80 2023-03-27T10:01:57.881 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.879+0000 7f8f4139a700 1 -- 172.21.15.23:0/3831956625 >> v2:172.21.15.23:3300/0 conn(0x7f8f3c15ce00 msgr2=0x7f8f3c0cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:01:57.881 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.879+0000 7f8f4139a700 1 --2- 172.21.15.23:0/3831956625 >> v2:172.21.15.23:3300/0 conn(0x7f8f3c15ce00 0x7f8f3c0cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:57.882 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.879+0000 7f8f4139a700 1 -- 172.21.15.23:0/3831956625 >> v1:172.21.15.112:6789/0 conn(0x7f8f3c152e80 legacy=0x7f8f3c0c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:57.882 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.879+0000 7f8f4139a700 1 -- 172.21.15.23:0/3831956625 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f3c159380 con 0x7f8f3c0c4eb0 2023-03-27T10:01:57.882 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.879+0000 7f8f4239c700 1 --2- 172.21.15.23:0/3831956625 >> v2:172.21.15.23:3300/0 conn(0x7f8f3c15ce00 0x7f8f3c0cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-27T10:01:57.882 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.880+0000 7f8f4139a700 1 --2- 172.21.15.23:0/3831956625 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f3c0c4eb0 0x7f8f3c15c8c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c009840 tx=0x7f8f2c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9322243300a43c3 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:01:57.882 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.880+0000 7f8f40b99700 1 -- 172.21.15.23:0/3831956625 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c011830 con 0x7f8f3c0c4eb0 2023-03-27T10:01:57.882 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.880+0000 7f8f40b99700 1 -- 172.21.15.23:0/3831956625 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f2c011990 con 0x7f8f3c0c4eb0 2023-03-27T10:01:57.882 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.880+0000 7f8f40b99700 1 -- 172.21.15.23:0/3831956625 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c011d20 con 0x7f8f3c0c4eb0 2023-03-27T10:01:57.883 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.880+0000 7f8f43dff700 1 -- 172.21.15.23:0/3831956625 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f3c0c4eb0 msgr2=0x7f8f3c15c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:57.883 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.880+0000 7f8f43dff700 1 --2- 172.21.15.23:0/3831956625 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f3c0c4eb0 0x7f8f3c15c8c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c009840 tx=0x7f8f2c009b50 comp rx=0 tx=0).stop 2023-03-27T10:01:57.883 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.880+0000 7f8f43dff700 1 -- 172.21.15.23:0/3831956625 shutdown_connections 2023-03-27T10:01:57.883 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.880+0000 7f8f43dff700 1 --2- 172.21.15.23:0/3831956625 >> v2:172.21.15.23:3300/0 conn(0x7f8f3c15ce00 0x7f8f3c0cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:57.884 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.880+0000 7f8f43dff700 1 --2- 172.21.15.23:0/3831956625 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f3c0c4eb0 0x7f8f3c15c8c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:57.884 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.880+0000 7f8f43dff700 1 -- 172.21.15.23:0/3831956625 >> 172.21.15.23:0/3831956625 conn(0x7f8f3c065ca0 msgr2=0x7f8f3c0718f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:01:57.884 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.881+0000 7f8f43dff700 1 -- 172.21.15.23:0/3831956625 shutdown_connections 2023-03-27T10:01:57.884 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.881+0000 7f8f43dff700 1 -- 172.21.15.23:0/3831956625 wait complete. 2023-03-27T10:01:57.886 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.882+0000 7f8f43dff700 1 Processor -- start 2023-03-27T10:01:57.886 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.882+0000 7f8f43dff700 1 -- start start 2023-03-27T10:01:57.887 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.883+0000 7f8f43dff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f8f3c0c4eb0 0x7f8f3c157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:57.887 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.883+0000 7f8f43dff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f3c152e80 0x7f8f3c145f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:57.887 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.883+0000 7f8f43dff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f3c15ce00 0x7f8f3c146450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:01:57.887 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.883+0000 7f8f43dff700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c1526e0 con 0x7f8f3c0c4eb0 2023-03-27T10:01:57.887 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.883+0000 7f8f43dff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c1ef040 con 0x7f8f3c15ce00 2023-03-27T10:01:57.887 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.883+0000 7f8f43dff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c146990 con 0x7f8f3c152e80 2023-03-27T10:01:57.887 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.883+0000 7f8f41b9b700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f8f3c0c4eb0 0x7f8f3c157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:57.887 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.883+0000 7f8f4139a700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f3c152e80 0x7f8f3c145f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:57.888 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.884+0000 7f8f41b9b700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f8f3c0c4eb0 0x7f8f3c157400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:57618/0 (socket says 172.21.15.23:57618) 2023-03-27T10:01:57.888 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.884+0000 7f8f41b9b700 1 -- 172.21.15.23:0/3558031071 learned_addr learned my addr 172.21.15.23:0/3558031071 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:01:57.888 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.884+0000 7f8f4239c700 1 --2- 172.21.15.23:0/3558031071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f3c15ce00 0x7f8f3c146450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:01:57.888 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.884+0000 7f8f4139a700 1 -- 172.21.15.23:0/3558031071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f3c15ce00 msgr2=0x7f8f3c146450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:57.888 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.884+0000 7f8f4139a700 1 --2- 172.21.15.23:0/3558031071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f3c15ce00 0x7f8f3c146450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:57.888 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.884+0000 7f8f4139a700 1 -- 172.21.15.23:0/3558031071 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f8f3c0c4eb0 msgr2=0x7f8f3c157400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:57.888 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.884+0000 7f8f4139a700 1 --2- 172.21.15.23:0/3558031071 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f8f3c0c4eb0 0x7f8f3c157400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:57.889 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.884+0000 7f8f4139a700 1 -- 172.21.15.23:0/3558031071 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f34009a70 con 0x7f8f3c152e80 2023-03-27T10:01:57.889 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.884+0000 7f8f4139a700 1 --2- 172.21.15.23:0/3558031071 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f3c152e80 0x7f8f3c145f10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c016000 tx=0x7f8f2c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:01:57.889 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.884+0000 7f8f327fc700 1 -- 172.21.15.23:0/3558031071 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c004550 con 0x7f8f3c152e80 2023-03-27T10:01:57.889 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.884+0000 7f8f327fc700 1 -- 172.21.15.23:0/3558031071 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f2c01e070 con 0x7f8f3c152e80 2023-03-27T10:01:57.889 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.884+0000 7f8f43dff700 1 -- 172.21.15.23:0/3558031071 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f2c0094a0 con 0x7f8f3c152e80 2023-03-27T10:01:57.889 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.885+0000 7f8f327fc700 1 -- 172.21.15.23:0/3558031071 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c0038c0 con 0x7f8f3c152e80 2023-03-27T10:01:57.889 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.885+0000 7f8f43dff700 1 -- 172.21.15.23:0/3558031071 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f3c159940 con 0x7f8f3c152e80 2023-03-27T10:01:57.890 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.885+0000 7f8f327fc700 1 -- 172.21.15.23:0/3558031071 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f8f2c00f050 con 0x7f8f3c152e80 2023-03-27T10:01:57.890 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.885+0000 7f8f327fc700 1 -- 172.21.15.23:0/3558031071 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f8f2c02ddf0 con 0x7f8f3c152e80 2023-03-27T10:01:57.890 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.886+0000 7f8f43dff700 1 -- 172.21.15.23:0/3558031071 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f3c159660 con 0x7f8f3c152e80 2023-03-27T10:01:57.890 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.888+0000 7f8f327fc700 1 -- 172.21.15.23:0/3558031071 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72400 (secure 0 0 0) 0x7f8f2c020070 con 0x7f8f3c152e80 2023-03-27T10:01:57.914 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-03-27T10:01:57.915+0000 7f105b615200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-03-27T10:01:57.961 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.961+0000 7f8f43dff700 1 -- 172.21.15.23:0/3558031071 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f3c15c3a0 con 0x7f8f3c152e80 2023-03-27T10:01:57.962 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.962+0000 7f8f327fc700 1 -- 172.21.15.23:0/3558031071 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f8f2c015020 con 0x7f8f3c152e80 2023-03-27T10:01:57.962 INFO:teuthology.misc.health.smithi023.stdout: 2023-03-27T10:01:57.962 INFO:teuthology.misc.health.smithi023.stdout:{"epoch":10,"fsid":"1d8d6e6e-0d21-4f51-91ee-d9434d4fb750","created":"2023-03-27T10:01:53.805778+0000","modified":"2023-03-27T10:01:56.475091+0000","last_up_change":"0.000000","last_in_change":"2023-03-27T10:01:56.475091+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"df086052-536b-4fb1-bcce-7aaf5a13911e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"03138001-a3fd-43fc-9797-a47643d9a3a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"964dd926-1160-44ef-aa34-b18965dc05f3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4ae5500f-cb24-44a8-a536-5c399c09e566","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"de228ab2-92b4-40bc-8dbd-8f2f4895dc22","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6c839571-b490-4e66-92ac-0d7301c2e677","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"871b1a2b-f0ed-4faf-8f9a-a5a0973e11c9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d8228353-34b5-4363-84d8-642512f5bc06","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-27T10:01:57.963 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.963+0000 7f8f43dff700 1 -- 172.21.15.23:0/3558031071 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f3c152e80 msgr2=0x7f8f3c145f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:01:57.963 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.963+0000 7f8f43dff700 1 --2- 172.21.15.23:0/3558031071 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f3c152e80 0x7f8f3c145f10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c016000 tx=0x7f8f2c004750 comp rx=0 tx=0).stop 2023-03-27T10:01:57.963 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.963+0000 7f8f43dff700 1 -- 172.21.15.23:0/3558031071 shutdown_connections 2023-03-27T10:01:57.963 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.963+0000 7f8f43dff700 1 --2- 172.21.15.23:0/3558031071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f3c15ce00 0x7f8f3c146450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:57.964 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.963+0000 7f8f43dff700 1 --2- 172.21.15.23:0/3558031071 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f8f3c0c4eb0 0x7f8f3c157400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:57.964 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.963+0000 7f8f43dff700 1 --2- 172.21.15.23:0/3558031071 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f3c152e80 0x7f8f3c145f10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:01:57.964 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.963+0000 7f8f43dff700 1 -- 172.21.15.23:0/3558031071 >> 172.21.15.23:0/3558031071 conn(0x7f8f3c065ca0 msgr2=0x7f8f3c14d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:01:57.964 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.963+0000 7f8f43dff700 1 -- 172.21.15.23:0/3558031071 shutdown_connections 2023-03-27T10:01:57.964 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:01:57.963+0000 7f8f43dff700 1 -- 172.21.15.23:0/3558031071 wait complete. 2023-03-27T10:01:57.977 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-03-27T10:01:58.029 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-03-27T10:01:58.030+0000 7f105b615200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-03-27T10:01:58.038 INFO:tasks.ceph.mgr.y.smithi023.stderr:2023-03-27T10:01:58.038+0000 7f6256375200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-03-27T10:01:58.173 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-03-27T10:01:58.173+0000 7f105b615200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-03-27T10:01:58.205 INFO:tasks.ceph.mgr.y.smithi023.stderr:2023-03-27T10:01:58.205+0000 7f6256375200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-03-27T10:01:58.272 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-03-27T10:01:58.272+0000 7f105b615200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-03-27T10:01:58.622 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-03-27T10:01:58.622+0000 7f105b615200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-03-27T10:01:58.633 INFO:tasks.ceph.mgr.y.smithi023.stderr:2023-03-27T10:01:58.634+0000 7f6256375200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-03-27T10:01:58.829 INFO:tasks.ceph.mgr.y.smithi023.stderr:2023-03-27T10:01:58.830+0000 7f6256375200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-03-27T10:01:58.980 INFO:tasks.ceph.mgr.y.smithi023.stderr:2023-03-27T10:01:58.981+0000 7f6256375200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-03-27T10:01:59.164 INFO:tasks.ceph.mgr.y.smithi023.stderr:2023-03-27T10:01:59.165+0000 7f6256375200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-03-27T10:01:59.199 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-03-27T10:01:59.200+0000 7f105b615200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-03-27T10:01:59.294 INFO:tasks.ceph.mgr.y.smithi023.stderr:2023-03-27T10:01:59.294+0000 7f6256375200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-03-27T10:01:59.719 INFO:tasks.ceph.mgr.y.smithi023.stderr:2023-03-27T10:01:59.719+0000 7f6256375200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-03-27T10:02:00.227 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-03-27T10:02:00.228+0000 7f101398e700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-03-27T10:02:00.227 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-03-27T10:02:00.228+0000 7f101398e700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-03-27T10:02:00.603 INFO:tasks.ceph.mgr.y.smithi023.stderr:2023-03-27T10:02:00.604+0000 7f6256375200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-03-27T10:02:00.676 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-03-27T10:02:00.677+0000 7f6692b4d580 -1 osd.6 0 log_to_monitors true 2023-03-27T10:02:00.708 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-03-27T10:02:00.709+0000 7f5f82a08580 -1 osd.5 0 log_to_monitors true 2023-03-27T10:02:00.733 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-03-27T10:02:00.734+0000 7fa44f917580 -1 osd.4 0 log_to_monitors true 2023-03-27T10:02:00.734 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-03-27T10:02:00.734+0000 7fd9215fc580 -1 osd.7 0 log_to_monitors true 2023-03-27T10:02:00.834 INFO:tasks.ceph.osd.1.smithi023.stderr:2023-03-27T10:02:00.835+0000 7efc17a6f580 -1 osd.1 0 log_to_monitors true 2023-03-27T10:02:00.884 INFO:tasks.ceph.osd.3.smithi023.stderr:2023-03-27T10:02:00.884+0000 7f58f27bc580 -1 osd.3 0 log_to_monitors true 2023-03-27T10:02:00.898 INFO:tasks.ceph.osd.2.smithi023.stderr:2023-03-27T10:02:00.899+0000 7f716e8d9580 -1 osd.2 0 log_to_monitors true 2023-03-27T10:02:01.089 INFO:tasks.ceph.osd.0.smithi023.stderr:2023-03-27T10:02:01.089+0000 7efdea2e7580 -1 osd.0 0 log_to_monitors true 2023-03-27T10:02:03.157 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-03-27T10:02:03.157+0000 7f5f76dc0700 -1 osd.5 0 waiting for initial osdmap 2023-03-27T10:02:03.157 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-03-27T10:02:03.158+0000 7f6686f05700 -1 osd.6 0 waiting for initial osdmap 2023-03-27T10:02:03.169 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-03-27T10:02:03.170+0000 7f5f6e99a700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-27T10:02:03.169 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-03-27T10:02:03.170+0000 7f667eadf700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-27T10:02:03.182 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-03-27T10:02:03.182+0000 7fa443ccf700 -1 osd.4 0 waiting for initial osdmap 2023-03-27T10:02:03.194 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-03-27T10:02:03.195+0000 7fa43b8a9700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-27T10:02:03.198 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-03-27T10:02:03.200+0000 7fd9159b4700 -1 osd.7 0 waiting for initial osdmap 2023-03-27T10:02:03.210 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-03-27T10:02:03.212+0000 7fd90d58e700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-27T10:02:03.599 INFO:tasks.ceph.osd.1.smithi023.stderr:2023-03-27T10:02:03.600+0000 7efc069f7700 -1 osd.1 0 waiting for initial osdmap 2023-03-27T10:02:03.613 INFO:tasks.ceph.osd.1.smithi023.stderr:2023-03-27T10:02:03.613+0000 7efbfe1fa700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-27T10:02:03.614 INFO:tasks.ceph.osd.2.smithi023.stderr:2023-03-27T10:02:03.614+0000 7f715d7f7700 -1 osd.2 0 waiting for initial osdmap 2023-03-27T10:02:03.632 INFO:tasks.ceph.osd.0.smithi023.stderr:2023-03-27T10:02:03.632+0000 7efdd91f7700 -1 osd.0 0 waiting for initial osdmap 2023-03-27T10:02:03.634 INFO:tasks.ceph.osd.2.smithi023.stderr:2023-03-27T10:02:03.633+0000 7f7154ffa700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-27T10:02:03.642 INFO:tasks.ceph.osd.3.smithi023.stderr:2023-03-27T10:02:03.642+0000 7f58e1df8700 -1 osd.3 0 waiting for initial osdmap 2023-03-27T10:02:03.649 INFO:tasks.ceph.osd.0.smithi023.stderr:2023-03-27T10:02:03.650+0000 7efdd09fa700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-27T10:02:03.658 INFO:tasks.ceph.osd.3.smithi023.stderr:2023-03-27T10:02:03.658+0000 7f58d8dfa700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-27T10:02:04.785 DEBUG:teuthology.orchestra.run.smithi023:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-03-27T10:02:04.965 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.966+0000 7fe3b13ff700 1 Processor -- start 2023-03-27T10:02:04.966 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.966+0000 7fe3b13ff700 1 -- start start 2023-03-27T10:02:04.966 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.967+0000 7fe3b13ff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe3ac0c5690 0x7fe3ac0ca740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:04.967 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.967+0000 7fe3b13ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe3ac0c5170 0x7fe3ac0cac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:04.967 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.967+0000 7fe3b13ff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3ac156a10 con 0x7fe3ac150bd0 2023-03-27T10:02:04.967 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.967+0000 7fe3b13ff700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3ac150fa0 con 0x7fe3ac0c5690 2023-03-27T10:02:04.967 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.967+0000 7fe3b13ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ac156770 con 0x7fe3ac0c5170 2023-03-27T10:02:04.969 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.967+0000 7fe3ab7fe700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe3ac0c5170 0x7fe3ac0cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:04.969 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.967+0000 7fe3aa7fc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe3ac0c5690 0x7fe3ac0ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:04.969 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.967+0000 7fe3aaffd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe3ac150bd0 0x7fe3ac156ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.23:50100/0 (socket says 172.21.15.23:50100) 2023-03-27T10:02:04.969 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.968+0000 7fe3aaffd700 1 -- 172.21.15.23:0/4144438956 learned_addr learned my addr 172.21.15.23:0/4144438956 (peer_addr_for_me v1:172.21.15.23:0/0) 2023-03-27T10:02:04.969 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.968+0000 7fe3a9ffb700 1 -- 172.21.15.23:0/4144438956 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2207311871 0 0) 0x7fe3ac156a10 con 0x7fe3ac150bd0 2023-03-27T10:02:04.969 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.968+0000 7fe3ab7fe700 1 -- 172.21.15.23:0/4144438956 >> v2:172.21.15.112:3300/0 conn(0x7fe3ac0c5690 msgr2=0x7fe3ac0ca740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:02:04.969 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.968+0000 7fe3ab7fe700 1 --2- 172.21.15.23:0/4144438956 >> v2:172.21.15.112:3300/0 conn(0x7fe3ac0c5690 0x7fe3ac0ca740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:04.970 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.968+0000 7fe3ab7fe700 1 -- 172.21.15.23:0/4144438956 >> v1:172.21.15.112:6789/0 conn(0x7fe3ac150bd0 legacy=0x7fe3ac156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:04.970 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.968+0000 7fe3ab7fe700 1 -- 172.21.15.23:0/4144438956 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3ac0c4770 con 0x7fe3ac0c5170 2023-03-27T10:02:04.970 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.968+0000 7fe3ab7fe700 1 --2- 172.21.15.23:0/4144438956 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe3ac0c5170 0x7fe3ac0cac80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4009d90 tx=0x7fe3a400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=193ecbc66e6dd66d server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:04.970 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.968+0000 7fe3a9ffb700 1 -- 172.21.15.23:0/4144438956 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a400e070 con 0x7fe3ac0c5170 2023-03-27T10:02:04.970 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.968+0000 7fe3a9ffb700 1 -- 172.21.15.23:0/4144438956 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a400bb20 con 0x7fe3ac0c5170 2023-03-27T10:02:04.970 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.968+0000 7fe3a9ffb700 1 -- 172.21.15.23:0/4144438956 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a4012430 con 0x7fe3ac0c5170 2023-03-27T10:02:04.970 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.969+0000 7fe3b13ff700 1 -- 172.21.15.23:0/4144438956 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe3ac0c5170 msgr2=0x7fe3ac0cac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:04.971 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.969+0000 7fe3b13ff700 1 --2- 172.21.15.23:0/4144438956 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe3ac0c5170 0x7fe3ac0cac80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4009d90 tx=0x7fe3a400b330 comp rx=0 tx=0).stop 2023-03-27T10:02:04.971 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.969+0000 7fe3b13ff700 1 -- 172.21.15.23:0/4144438956 shutdown_connections 2023-03-27T10:02:04.971 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.969+0000 7fe3b13ff700 1 --2- 172.21.15.23:0/4144438956 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe3ac0c5170 0x7fe3ac0cac80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:04.971 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.969+0000 7fe3b13ff700 1 --2- 172.21.15.23:0/4144438956 >> v2:172.21.15.112:3300/0 conn(0x7fe3ac0c5690 0x7fe3ac0ca740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:04.971 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.969+0000 7fe3b13ff700 1 -- 172.21.15.23:0/4144438956 >> 172.21.15.23:0/4144438956 conn(0x7fe3ac06a1c0 msgr2=0x7fe3ac06a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:04.971 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.969+0000 7fe3b13ff700 1 -- 172.21.15.23:0/4144438956 shutdown_connections 2023-03-27T10:02:04.972 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.969+0000 7fe3b13ff700 1 -- 172.21.15.23:0/4144438956 wait complete. 2023-03-27T10:02:04.975 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.971+0000 7fe3b13ff700 1 Processor -- start 2023-03-27T10:02:04.975 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.971+0000 7fe3b13ff700 1 -- start start 2023-03-27T10:02:04.975 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.972+0000 7fe3b13ff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fe3ac0c5170 0x7fe3ac145540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:04.975 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.972+0000 7fe3b13ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe3ac150bd0 0x7fe3ac1461b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:04.975 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.972+0000 7fe3b13ff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3ac1466f0 0x7fe3ac1ede70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:04.976 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.972+0000 7fe3b13ff700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ac146b80 con 0x7fe3ac0c5170 2023-03-27T10:02:04.976 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.972+0000 7fe3b13ff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ac146e60 con 0x7fe3ac1466f0 2023-03-27T10:02:04.976 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.972+0000 7fe3b13ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ac147140 con 0x7fe3ac150bd0 2023-03-27T10:02:04.976 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.972+0000 7fe3aaffd700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fe3ac0c5170 0x7fe3ac145540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:04.976 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.972+0000 7fe3aaffd700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fe3ac0c5170 0x7fe3ac145540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:57670/0 (socket says 172.21.15.23:57670) 2023-03-27T10:02:04.976 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.972+0000 7fe3aaffd700 1 -- 172.21.15.23:0/2908484415 learned_addr learned my addr 172.21.15.23:0/2908484415 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:04.976 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.972+0000 7fe3ab7fe700 1 --2- 172.21.15.23:0/2908484415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3ac1466f0 0x7fe3ac1ede70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:04.977 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.972+0000 7fe3aa7fc700 1 --2- 172.21.15.23:0/2908484415 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe3ac150bd0 0x7fe3ac1461b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:04.977 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.973+0000 7fe3aaffd700 1 -- 172.21.15.23:0/2908484415 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe3ac150bd0 msgr2=0x7fe3ac1461b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:04.977 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.973+0000 7fe3aaffd700 1 --2- 172.21.15.23:0/2908484415 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe3ac150bd0 0x7fe3ac1461b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:04.977 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.973+0000 7fe3aaffd700 1 -- 172.21.15.23:0/2908484415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3ac1466f0 msgr2=0x7fe3ac1ede70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:04.977 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.973+0000 7fe3aaffd700 1 --2- 172.21.15.23:0/2908484415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3ac1466f0 0x7fe3ac1ede70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:04.977 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.973+0000 7fe3aaffd700 1 -- 172.21.15.23:0/2908484415 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a4009a70 con 0x7fe3ac0c5170 2023-03-27T10:02:04.977 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.973+0000 7fe3aaffd700 1 --2- 172.21.15.23:0/2908484415 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fe3ac0c5170 0x7fe3ac145540 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe39c00c550 tx=0x7fe39c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:04.978 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.973+0000 7fe3937fe700 1 -- 172.21.15.23:0/2908484415 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39c004c80 con 0x7fe3ac0c5170 2023-03-27T10:02:04.978 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.973+0000 7fe3937fe700 1 -- 172.21.15.23:0/2908484415 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe39c004e00 con 0x7fe3ac0c5170 2023-03-27T10:02:04.978 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.973+0000 7fe3937fe700 1 -- 172.21.15.23:0/2908484415 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39c007310 con 0x7fe3ac0c5170 2023-03-27T10:02:04.978 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.973+0000 7fe3b13ff700 1 -- 172.21.15.23:0/2908484415 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3ac0c6180 con 0x7fe3ac0c5170 2023-03-27T10:02:04.978 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.973+0000 7fe3b13ff700 1 -- 172.21.15.23:0/2908484415 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3ac0c67d0 con 0x7fe3ac0c5170 2023-03-27T10:02:04.978 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.974+0000 7fe3b13ff700 1 -- 172.21.15.23:0/2908484415 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3ac0c4a10 con 0x7fe3ac0c5170 2023-03-27T10:02:04.978 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.975+0000 7fe3937fe700 1 -- 172.21.15.23:0/2908484415 <== mon.0 v2:172.21.15.23:3300/0 4 ==== mgrmap(e 5) v1 ==== 85073+0+0 (secure 0 0 0) 0x7fe39c01a020 con 0x7fe3ac0c5170 2023-03-27T10:02:04.979 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.976+0000 7fe3937fe700 1 --2- 172.21.15.23:0/2908484415 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fe398066de0 0x7fe398069290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:04.979 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.976+0000 7fe3937fe700 1 -- 172.21.15.23:0/2908484415 <== mon.0 v2:172.21.15.23:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7fe39c0836f0 con 0x7fe3ac0c5170 2023-03-27T10:02:04.979 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.976+0000 7fe3aa7fc700 1 --2- 172.21.15.23:0/2908484415 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fe398066de0 0x7fe398069290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:04.980 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.977+0000 7fe3aa7fc700 1 --2- 172.21.15.23:0/2908484415 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fe398066de0 0x7fe398069290 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac1ed290 tx=0x7fe3a0009210 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:04.980 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:04.981+0000 7fe3937fe700 1 -- 172.21.15.23:0/2908484415 <== mon.0 v2:172.21.15.23:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173280 (secure 0 0 0) 0x7fe39c050fb0 con 0x7fe3ac0c5170 2023-03-27T10:02:05.182 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:05.183+0000 7fe3b13ff700 1 -- 172.21.15.23:0/2908484415 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3ac072710 con 0x7fe3ac0c5170 2023-03-27T10:02:05.183 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:05.184+0000 7fe3937fe700 1 -- 172.21.15.23:0/2908484415 <== mon.0 v2:172.21.15.23:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10488 (secure 0 0 0) 0x7fe39c0545d0 con 0x7fe3ac0c5170 2023-03-27T10:02:05.184 INFO:teuthology.misc.health.smithi023.stdout: 2023-03-27T10:02:05.184 INFO:teuthology.misc.health.smithi023.stdout:{"epoch":13,"fsid":"1d8d6e6e-0d21-4f51-91ee-d9434d4fb750","created":"2023-03-27T10:01:53.805778+0000","modified":"2023-03-27T10:02:04.172266+0000","last_up_change":"2023-03-27T10:02:04.172266+0000","last_in_change":"2023-03-27T10:01:56.475091+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"df086052-536b-4fb1-bcce-7aaf5a13911e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6800","nonce":96642},{"type":"v1","addr":"172.21.15.23:6801","nonce":96642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6802","nonce":96642},{"type":"v1","addr":"172.21.15.23:6803","nonce":96642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6806","nonce":96642},{"type":"v1","addr":"172.21.15.23:6807","nonce":96642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6804","nonce":96642},{"type":"v1","addr":"172.21.15.23:6805","nonce":96642}]},"public_addr":"172.21.15.23:6801/96642","cluster_addr":"172.21.15.23:6803/96642","heartbeat_back_addr":"172.21.15.23:6807/96642","heartbeat_front_addr":"172.21.15.23:6805/96642","state":["exists","up"]},{"osd":1,"uuid":"03138001-a3fd-43fc-9797-a47643d9a3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6808","nonce":96643},{"type":"v1","addr":"172.21.15.23:6809","nonce":96643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6810","nonce":96643},{"type":"v1","addr":"172.21.15.23:6811","nonce":96643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6814","nonce":96643},{"type":"v1","addr":"172.21.15.23:6815","nonce":96643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6812","nonce":96643},{"type":"v1","addr":"172.21.15.23:6813","nonce":96643}]},"public_addr":"172.21.15.23:6809/96643","cluster_addr":"172.21.15.23:6811/96643","heartbeat_back_addr":"172.21.15.23:6815/96643","heartbeat_front_addr":"172.21.15.23:6813/96643","state":["exists","up"]},{"osd":2,"uuid":"964dd926-1160-44ef-aa34-b18965dc05f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6817","nonce":96644},{"type":"v1","addr":"172.21.15.23:6819","nonce":96644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6820","nonce":96644},{"type":"v1","addr":"172.21.15.23:6822","nonce":96644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6827","nonce":96644},{"type":"v1","addr":"172.21.15.23:6829","nonce":96644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6824","nonce":96644},{"type":"v1","addr":"172.21.15.23:6825","nonce":96644}]},"public_addr":"172.21.15.23:6819/96644","cluster_addr":"172.21.15.23:6822/96644","heartbeat_back_addr":"172.21.15.23:6829/96644","heartbeat_front_addr":"172.21.15.23:6825/96644","state":["exists","up"]},{"osd":3,"uuid":"4ae5500f-cb24-44a8-a536-5c399c09e566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6816","nonce":96645},{"type":"v1","addr":"172.21.15.23:6818","nonce":96645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6821","nonce":96645},{"type":"v1","addr":"172.21.15.23:6823","nonce":96645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6830","nonce":96645},{"type":"v1","addr":"172.21.15.23:6831","nonce":96645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6826","nonce":96645},{"type":"v1","addr":"172.21.15.23:6828","nonce":96645}]},"public_addr":"172.21.15.23:6818/96645","cluster_addr":"172.21.15.23:6823/96645","heartbeat_back_addr":"172.21.15.23:6831/96645","heartbeat_front_addr":"172.21.15.23:6828/96645","state":["exists","up"]},{"osd":4,"uuid":"de228ab2-92b4-40bc-8dbd-8f2f4895dc22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":127986},{"type":"v1","addr":"172.21.15.112:6809","nonce":127986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":127986},{"type":"v1","addr":"172.21.15.112:6811","nonce":127986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6815","nonce":127986},{"type":"v1","addr":"172.21.15.112:6818","nonce":127986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":127986},{"type":"v1","addr":"172.21.15.112:6813","nonce":127986}]},"public_addr":"172.21.15.112:6809/127986","cluster_addr":"172.21.15.112:6811/127986","heartbeat_back_addr":"172.21.15.112:6818/127986","heartbeat_front_addr":"172.21.15.112:6813/127986","state":["exists","up"]},{"osd":5,"uuid":"6c839571-b490-4e66-92ac-0d7301c2e677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":127987},{"type":"v1","addr":"172.21.15.112:6801","nonce":127987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":127987},{"type":"v1","addr":"172.21.15.112:6803","nonce":127987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":127987},{"type":"v1","addr":"172.21.15.112:6807","nonce":127987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":127987},{"type":"v1","addr":"172.21.15.112:6805","nonce":127987}]},"public_addr":"172.21.15.112:6801/127987","cluster_addr":"172.21.15.112:6803/127987","heartbeat_back_addr":"172.21.15.112:6807/127987","heartbeat_front_addr":"172.21.15.112:6805/127987","state":["exists","up"]},{"osd":6,"uuid":"871b1a2b-f0ed-4faf-8f9a-a5a0973e11c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":127988},{"type":"v1","addr":"172.21.15.112:6816","nonce":127988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":127988},{"type":"v1","addr":"172.21.15.112:6819","nonce":127988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":127988},{"type":"v1","addr":"172.21.15.112:6823","nonce":127988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":127988},{"type":"v1","addr":"172.21.15.112:6821","nonce":127988}]},"public_addr":"172.21.15.112:6816/127988","cluster_addr":"172.21.15.112:6819/127988","heartbeat_back_addr":"172.21.15.112:6823/127988","heartbeat_front_addr":"172.21.15.112:6821/127988","state":["exists","up"]},{"osd":7,"uuid":"d8228353-34b5-4363-84d8-642512f5bc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":128009},{"type":"v1","addr":"172.21.15.112:6825","nonce":128009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":128009},{"type":"v1","addr":"172.21.15.112:6827","nonce":128009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":128009},{"type":"v1","addr":"172.21.15.112:6831","nonce":128009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":128009},{"type":"v1","addr":"172.21.15.112:6829","nonce":128009}]},"public_addr":"172.21.15.112:6825/128009","cluster_addr":"172.21.15.112:6827/128009","heartbeat_back_addr":"172.21.15.112:6831/128009","heartbeat_front_addr":"172.21.15.112:6829/128009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-27T10:02:05.186 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:05.186+0000 7fe3b13ff700 1 -- 172.21.15.23:0/2908484415 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fe398066de0 msgr2=0x7fe398069290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:05.186 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:05.186+0000 7fe3b13ff700 1 --2- 172.21.15.23:0/2908484415 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fe398066de0 0x7fe398069290 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac1ed290 tx=0x7fe3a0009210 comp rx=0 tx=0).stop 2023-03-27T10:02:05.187 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:05.186+0000 7fe3b13ff700 1 -- 172.21.15.23:0/2908484415 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fe3ac0c5170 msgr2=0x7fe3ac145540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:05.187 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:05.186+0000 7fe3b13ff700 1 --2- 172.21.15.23:0/2908484415 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fe3ac0c5170 0x7fe3ac145540 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe39c00c550 tx=0x7fe39c00c860 comp rx=0 tx=0).stop 2023-03-27T10:02:05.187 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:05.186+0000 7fe3b13ff700 1 -- 172.21.15.23:0/2908484415 shutdown_connections 2023-03-27T10:02:05.187 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:05.186+0000 7fe3b13ff700 1 --2- 172.21.15.23:0/2908484415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3ac1466f0 0x7fe3ac1ede70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:05.187 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:05.186+0000 7fe3b13ff700 1 --2- 172.21.15.23:0/2908484415 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fe3ac0c5170 0x7fe3ac145540 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:05.187 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:05.186+0000 7fe3b13ff700 1 --2- 172.21.15.23:0/2908484415 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fe398066de0 0x7fe398069290 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:05.187 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:05.186+0000 7fe3b13ff700 1 --2- 172.21.15.23:0/2908484415 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe3ac150bd0 0x7fe3ac1461b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:05.187 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:05.186+0000 7fe3b13ff700 1 -- 172.21.15.23:0/2908484415 >> 172.21.15.23:0/2908484415 conn(0x7fe3ac06a1c0 msgr2=0x7fe3ac1550d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:05.188 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:05.186+0000 7fe3b13ff700 1 -- 172.21.15.23:0/2908484415 shutdown_connections 2023-03-27T10:02:05.188 INFO:teuthology.misc.health.smithi023.stderr:2023-03-27T10:02:05.186+0000 7fe3b13ff700 1 -- 172.21.15.23:0/2908484415 wait complete. 2023-03-27T10:02:05.201 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-03-27T10:02:05.201 INFO:tasks.ceph:Creating RBD pool 2023-03-27T10:02:05.202 DEBUG:teuthology.orchestra.run.smithi023:> sudo ceph --cluster ceph osd pool create rbd 8 2023-03-27T10:02:05.385 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.386+0000 7fb3725ff700 1 Processor -- start 2023-03-27T10:02:05.386 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.386+0000 7fb3725ff700 1 -- start start 2023-03-27T10:02:05.387 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.387+0000 7fb3725ff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb36c0c6990 0x7fb36c0c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:05.387 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.387+0000 7fb3725ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb36c0c9520 0x7fb36c0c8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:05.387 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.387+0000 7fb3725ff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb36c0ca730 con 0x7fb36c06b440 2023-03-27T10:02:05.387 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.387+0000 7fb3725ff700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb36c0c72a0 con 0x7fb36c0c6990 2023-03-27T10:02:05.387 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.387+0000 7fb3725ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c0cf9f0 con 0x7fb36c0c9520 2023-03-27T10:02:05.387 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.387+0000 7fb36b7fe700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb36c0c9520 0x7fb36c0c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:05.387 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.387+0000 7fb370b9c700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fb36c06b440 0x7fb36c06b880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.23:50106/0 (socket says 172.21.15.23:50106) 2023-03-27T10:02:05.388 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.387+0000 7fb370b9c700 1 -- 172.21.15.23:0/2274775820 learned_addr learned my addr 172.21.15.23:0/2274775820 (peer_addr_for_me v1:172.21.15.23:0/0) 2023-03-27T10:02:05.388 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.387+0000 7fb36bfff700 1 --2- 172.21.15.23:0/2274775820 >> v2:172.21.15.112:3300/0 conn(0x7fb36c0c6990 0x7fb36c0c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:05.389 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.388+0000 7fb36b7fe700 1 -- 172.21.15.23:0/2274775820 >> v2:172.21.15.112:3300/0 conn(0x7fb36c0c6990 msgr2=0x7fb36c0c6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:02:05.389 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.388+0000 7fb36b7fe700 1 --2- 172.21.15.23:0/2274775820 >> v2:172.21.15.112:3300/0 conn(0x7fb36c0c6990 0x7fb36c0c6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:05.389 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.388+0000 7fb36b7fe700 1 -- 172.21.15.23:0/2274775820 >> v1:172.21.15.112:6789/0 conn(0x7fb36c06b440 legacy=0x7fb36c06b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:05.389 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.388+0000 7fb36b7fe700 1 -- 172.21.15.23:0/2274775820 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb36c158a00 con 0x7fb36c0c9520 2023-03-27T10:02:05.390 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.388+0000 7fb36affd700 1 -- 172.21.15.23:0/2274775820 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2241383344 0 0) 0x7fb36c0ca730 con 0x7fb36c06b440 2023-03-27T10:02:05.390 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.388+0000 7fb36b7fe700 1 --2- 172.21.15.23:0/2274775820 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb36c0c9520 0x7fb36c0c8820 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb360009840 tx=0x7fb360009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5ba4b827c80f688 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:05.391 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.388+0000 7fb36affd700 1 -- 172.21.15.23:0/2274775820 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb360011830 con 0x7fb36c0c9520 2023-03-27T10:02:05.391 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.388+0000 7fb36affd700 1 -- 172.21.15.23:0/2274775820 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb360011990 con 0x7fb36c0c9520 2023-03-27T10:02:05.391 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.388+0000 7fb36affd700 1 -- 172.21.15.23:0/2274775820 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb360011d20 con 0x7fb36c0c9520 2023-03-27T10:02:05.391 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.389+0000 7fb3725ff700 1 -- 172.21.15.23:0/2274775820 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb36c0c9520 msgr2=0x7fb36c0c8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:05.391 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.389+0000 7fb3725ff700 1 --2- 172.21.15.23:0/2274775820 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb36c0c9520 0x7fb36c0c8820 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb360009840 tx=0x7fb360009b50 comp rx=0 tx=0).stop 2023-03-27T10:02:05.391 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.389+0000 7fb3725ff700 1 -- 172.21.15.23:0/2274775820 shutdown_connections 2023-03-27T10:02:05.392 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.389+0000 7fb3725ff700 1 --2- 172.21.15.23:0/2274775820 >> v2:172.21.15.112:3300/0 conn(0x7fb36c0c6990 0x7fb36c0c6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:05.392 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.389+0000 7fb3725ff700 1 --2- 172.21.15.23:0/2274775820 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb36c0c9520 0x7fb36c0c8820 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:05.392 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.389+0000 7fb3725ff700 1 -- 172.21.15.23:0/2274775820 >> 172.21.15.23:0/2274775820 conn(0x7fb36c071920 msgr2=0x7fb36c071cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:05.392 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.389+0000 7fb3725ff700 1 -- 172.21.15.23:0/2274775820 shutdown_connections 2023-03-27T10:02:05.392 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.389+0000 7fb3725ff700 1 -- 172.21.15.23:0/2274775820 wait complete. 2023-03-27T10:02:05.394 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.390+0000 7fb3725ff700 1 Processor -- start 2023-03-27T10:02:05.394 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.391+0000 7fb3725ff700 1 -- start start 2023-03-27T10:02:05.394 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.391+0000 7fb3725ff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fb36c06b440 0x7fb36c14db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:05.395 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.391+0000 7fb3725ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb36c0c6990 0x7fb36c1f22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:05.395 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.392+0000 7fb3725ff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb36c0c9520 0x7fb36c1556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:05.395 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.392+0000 7fb3725ff700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c1504c0 con 0x7fb36c06b440 2023-03-27T10:02:05.395 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.392+0000 7fb3725ff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c150760 con 0x7fb36c0c9520 2023-03-27T10:02:05.395 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.392+0000 7fb3725ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c14e0c0 con 0x7fb36c0c6990 2023-03-27T10:02:05.395 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.392+0000 7fb36bfff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fb36c06b440 0x7fb36c14db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:05.395 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.392+0000 7fb36b7fe700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb36c0c6990 0x7fb36c1f22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:05.396 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.392+0000 7fb36bfff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fb36c06b440 0x7fb36c14db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:57680/0 (socket says 172.21.15.23:57680) 2023-03-27T10:02:05.396 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.392+0000 7fb36bfff700 1 -- 172.21.15.23:0/125608635 learned_addr learned my addr 172.21.15.23:0/125608635 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:05.396 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.392+0000 7fb370b9c700 1 --2- 172.21.15.23:0/125608635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb36c0c9520 0x7fb36c1556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:05.396 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.392+0000 7fb36b7fe700 1 -- 172.21.15.23:0/125608635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb36c0c9520 msgr2=0x7fb36c1556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:05.396 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.392+0000 7fb36b7fe700 1 --2- 172.21.15.23:0/125608635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb36c0c9520 0x7fb36c1556e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:05.396 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.392+0000 7fb36b7fe700 1 -- 172.21.15.23:0/125608635 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fb36c06b440 msgr2=0x7fb36c14db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:05.396 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.392+0000 7fb36b7fe700 1 --2- 172.21.15.23:0/125608635 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fb36c06b440 0x7fb36c14db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:05.396 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.392+0000 7fb36b7fe700 1 -- 172.21.15.23:0/125608635 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3600094a0 con 0x7fb36c0c6990 2023-03-27T10:02:05.397 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.392+0000 7fb36b7fe700 1 --2- 172.21.15.23:0/125608635 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb36c0c6990 0x7fb36c1f22f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb360016000 tx=0x7fb360004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:05.397 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.393+0000 7fb368ff9700 1 -- 172.21.15.23:0/125608635 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3600036a0 con 0x7fb36c0c6990 2023-03-27T10:02:05.397 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.393+0000 7fb368ff9700 1 -- 172.21.15.23:0/125608635 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb36001f070 con 0x7fb36c0c6990 2023-03-27T10:02:05.397 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.393+0000 7fb3725ff700 1 -- 172.21.15.23:0/125608635 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb36c158fc0 con 0x7fb36c0c6990 2023-03-27T10:02:05.397 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.393+0000 7fb368ff9700 1 -- 172.21.15.23:0/125608635 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb360003a80 con 0x7fb36c0c6990 2023-03-27T10:02:05.397 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.393+0000 7fb3725ff700 1 -- 172.21.15.23:0/125608635 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb36c158ce0 con 0x7fb36c0c6990 2023-03-27T10:02:05.397 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.394+0000 7fb3725ff700 1 -- 172.21.15.23:0/125608635 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb36c1592a0 con 0x7fb36c0c6990 2023-03-27T10:02:05.398 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.395+0000 7fb368ff9700 1 -- 172.21.15.23:0/125608635 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) v1 ==== 85073+0+0 (secure 0 0 0) 0x7fb36000f050 con 0x7fb36c0c6990 2023-03-27T10:02:05.398 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.395+0000 7fb368ff9700 1 --2- 172.21.15.23:0/125608635 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fb358066e10 0x7fb3580692c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:05.398 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.396+0000 7fb368ff9700 1 -- 172.21.15.23:0/125608635 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7fb360021070 con 0x7fb36c0c6990 2023-03-27T10:02:05.399 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.396+0000 7fb36bfff700 1 --2- 172.21.15.23:0/125608635 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fb358066e10 0x7fb3580692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:05.399 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.396+0000 7fb36bfff700 1 --2- 172.21.15.23:0/125608635 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fb358066e10 0x7fb3580692c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb35c009ca0 tx=0x7fb35c009210 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:05.400 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.401+0000 7fb368ff9700 1 -- 172.21.15.23:0/125608635 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173280 (secure 0 0 0) 0x7fb360052be0 con 0x7fb36c0c6990 2023-03-27T10:02:05.602 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.602+0000 7fb3725ff700 1 -- 172.21.15.23:0/125608635 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fb36c0ceba0 con 0x7fb36c0c6990 2023-03-27T10:02:05.654 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.655+0000 7fb368ff9700 1 -- 172.21.15.23:0/125608635 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v14) v1 ==== 109+0+0 (secure 0 0 0) 0x7fb36002b0a0 con 0x7fb36c0c6990 2023-03-27T10:02:05.658 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.658+0000 7fb3725ff700 1 -- 172.21.15.23:0/125608635 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fb358066e10 msgr2=0x7fb3580692c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:05.658 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.658+0000 7fb3725ff700 1 --2- 172.21.15.23:0/125608635 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fb358066e10 0x7fb3580692c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb35c009ca0 tx=0x7fb35c009210 comp rx=0 tx=0).stop 2023-03-27T10:02:05.659 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.658+0000 7fb3725ff700 1 -- 172.21.15.23:0/125608635 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb36c0c6990 msgr2=0x7fb36c1f22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:05.659 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.658+0000 7fb3725ff700 1 --2- 172.21.15.23:0/125608635 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb36c0c6990 0x7fb36c1f22f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb360016000 tx=0x7fb360004750 comp rx=0 tx=0).stop 2023-03-27T10:02:05.659 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.658+0000 7fb3725ff700 1 -- 172.21.15.23:0/125608635 shutdown_connections 2023-03-27T10:02:05.660 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.658+0000 7fb3725ff700 1 --2- 172.21.15.23:0/125608635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb36c0c9520 0x7fb36c1556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:05.660 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.658+0000 7fb3725ff700 1 --2- 172.21.15.23:0/125608635 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fb36c06b440 0x7fb36c14db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:05.660 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.658+0000 7fb3725ff700 1 --2- 172.21.15.23:0/125608635 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fb358066e10 0x7fb3580692c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:05.660 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.658+0000 7fb3725ff700 1 --2- 172.21.15.23:0/125608635 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb36c0c6990 0x7fb36c1f22f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:05.661 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.658+0000 7fb3725ff700 1 -- 172.21.15.23:0/125608635 >> 172.21.15.23:0/125608635 conn(0x7fb36c071920 msgr2=0x7fb36c156080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:05.661 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.658+0000 7fb3725ff700 1 -- 172.21.15.23:0/125608635 shutdown_connections 2023-03-27T10:02:05.661 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.658+0000 7fb3725ff700 1 -- 172.21.15.23:0/125608635 wait complete. 2023-03-27T10:02:05.662 INFO:teuthology.orchestra.run.smithi023.stderr:pool 'rbd' created 2023-03-27T10:02:05.674 DEBUG:teuthology.orchestra.run.smithi023:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-03-27T10:02:05.859 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.860+0000 7f4f91dff700 1 Processor -- start 2023-03-27T10:02:05.860 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.860+0000 7f4f91dff700 1 -- start start 2023-03-27T10:02:05.861 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.861+0000 7f4f91dff700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f4f8c06b0b0 0x7f4f8c158890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:05.861 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.861+0000 7f4f91dff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4f8c06b480 0x7f4f8c0c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:05.861 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.861+0000 7f4f91dff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4f8c0c4640 0x7f4f8c0c4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:05.861 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.861+0000 7f4f91dff700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f8c1598f0 con 0x7f4f8c06b0b0 2023-03-27T10:02:05.861 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.861+0000 7f4f91dff700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f8c066b70 con 0x7f4f8c0c4640 2023-03-27T10:02:05.862 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.861+0000 7f4f91dff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8c15a270 con 0x7f4f8c06b480 2023-03-27T10:02:05.862 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.861+0000 7f4f8b7fe700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f4f8c06b0b0 0x7f4f8c158890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:05.862 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.861+0000 7f4f8b7fe700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f4f8c06b0b0 0x7f4f8c158890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:57690/0 (socket says 172.21.15.23:57690) 2023-03-27T10:02:05.862 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.861+0000 7f4f8b7fe700 1 -- 172.21.15.23:0/3181607845 learned_addr learned my addr 172.21.15.23:0/3181607845 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:05.862 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.861+0000 7f4f8affd700 1 --2- 172.21.15.23:0/3181607845 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4f8c06b480 0x7f4f8c0c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:05.862 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.862+0000 7f4f8bfff700 1 --2- 172.21.15.23:0/3181607845 >> v2:172.21.15.112:3300/0 conn(0x7f4f8c0c4640 0x7f4f8c0c4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:05.863 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.862+0000 7f4f8b7fe700 1 -- 172.21.15.23:0/3181607845 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4f8c06b480 msgr2=0x7f4f8c0c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:02:05.864 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.862+0000 7f4f8b7fe700 1 --2- 172.21.15.23:0/3181607845 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4f8c06b480 0x7f4f8c0c94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:05.864 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.862+0000 7f4f8b7fe700 1 -- 172.21.15.23:0/3181607845 >> v2:172.21.15.112:3300/0 conn(0x7f4f8c0c4640 msgr2=0x7f4f8c0c4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:02:05.864 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.862+0000 7f4f8b7fe700 1 --2- 172.21.15.23:0/3181607845 >> v2:172.21.15.112:3300/0 conn(0x7f4f8c0c4640 0x7f4f8c0c4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:05.864 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.862+0000 7f4f8b7fe700 1 -- 172.21.15.23:0/3181607845 --> v2:172.21.15.23:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f8c1590b0 con 0x7f4f8c06b0b0 2023-03-27T10:02:05.864 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.862+0000 7f4f8b7fe700 1 --2- 172.21.15.23:0/3181607845 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f4f8c06b0b0 0x7f4f8c158890 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c066940 tx=0x7f4f7c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19df8c5a5f713dda server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:05.864 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.862+0000 7f4f8a7fc700 1 -- 172.21.15.23:0/3181607845 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c011830 con 0x7f4f8c06b0b0 2023-03-27T10:02:05.865 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.862+0000 7f4f8a7fc700 1 -- 172.21.15.23:0/3181607845 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f7c011990 con 0x7f4f8c06b0b0 2023-03-27T10:02:05.865 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.862+0000 7f4f8a7fc700 1 -- 172.21.15.23:0/3181607845 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c011d20 con 0x7f4f8c06b0b0 2023-03-27T10:02:05.865 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.863+0000 7f4f91dff700 1 -- 172.21.15.23:0/3181607845 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f4f8c06b0b0 msgr2=0x7f4f8c158890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:05.865 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.863+0000 7f4f91dff700 1 --2- 172.21.15.23:0/3181607845 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f4f8c06b0b0 0x7f4f8c158890 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c066940 tx=0x7f4f7c009bd0 comp rx=0 tx=0).stop 2023-03-27T10:02:05.865 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.863+0000 7f4f91dff700 1 -- 172.21.15.23:0/3181607845 shutdown_connections 2023-03-27T10:02:05.865 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.863+0000 7f4f91dff700 1 --2- 172.21.15.23:0/3181607845 >> v2:172.21.15.112:3300/0 conn(0x7f4f8c0c4640 0x7f4f8c0c4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:05.865 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.863+0000 7f4f91dff700 1 --2- 172.21.15.23:0/3181607845 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f4f8c06b0b0 0x7f4f8c158890 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:05.866 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.863+0000 7f4f91dff700 1 --2- 172.21.15.23:0/3181607845 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4f8c06b480 0x7f4f8c0c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:05.866 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.863+0000 7f4f91dff700 1 -- 172.21.15.23:0/3181607845 >> 172.21.15.23:0/3181607845 conn(0x7f4f8c06beb0 msgr2=0x7f4f8c06c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:05.866 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.863+0000 7f4f91dff700 1 -- 172.21.15.23:0/3181607845 shutdown_connections 2023-03-27T10:02:05.866 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.863+0000 7f4f91dff700 1 -- 172.21.15.23:0/3181607845 wait complete. 2023-03-27T10:02:05.867 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.865+0000 7f4f91dff700 1 Processor -- start 2023-03-27T10:02:05.868 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.865+0000 7f4f91dff700 1 -- start start 2023-03-27T10:02:05.868 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.866+0000 7f4f91dff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f8c06b0b0 0x7f4f8c1f2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:05.868 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.866+0000 7f4f91dff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4f8c06b480 0x7f4f8c0b3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:05.868 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.866+0000 7f4f91dff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f4f8c0c4640 0x7f4f8c150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:05.868 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.866+0000 7f4f91dff700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8c1ee1a0 con 0x7f4f8c0c4640 2023-03-27T10:02:05.868 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.866+0000 7f4f91dff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8c1eea00 con 0x7f4f8c06b0b0 2023-03-27T10:02:05.869 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.866+0000 7f4f91dff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8c153180 con 0x7f4f8c06b480 2023-03-27T10:02:05.869 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.866+0000 7f4f8affd700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4f8c06b480 0x7f4f8c0b3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:05.869 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.866+0000 7f4f8b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f8c06b0b0 0x7f4f8c1f2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:05.869 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.866+0000 7f4f8bfff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f4f8c0c4640 0x7f4f8c150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:05.869 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.866+0000 7f4f8affd700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4f8c06b480 0x7f4f8c0b3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.23:36002/0 (socket says 172.21.15.23:36002) 2023-03-27T10:02:05.869 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.866+0000 7f4f8b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f8c06b0b0 0x7f4f8c1f2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.23:52200/0 (socket says 172.21.15.23:52200) 2023-03-27T10:02:05.869 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.866+0000 7f4f8affd700 1 -- 172.21.15.23:0/3286150518 learned_addr learned my addr 172.21.15.23:0/3286150518 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:05.870 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.867+0000 7f4f8bfff700 1 -- 172.21.15.23:0/3286150518 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4f8c06b480 msgr2=0x7f4f8c0b3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:05.870 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.867+0000 7f4f8bfff700 1 --2- 172.21.15.23:0/3286150518 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4f8c06b480 0x7f4f8c0b3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:05.870 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.867+0000 7f4f8bfff700 1 -- 172.21.15.23:0/3286150518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f8c06b0b0 msgr2=0x7f4f8c1f2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:05.870 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.867+0000 7f4f8bfff700 1 --2- 172.21.15.23:0/3286150518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f8c06b0b0 0x7f4f8c1f2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:05.870 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.867+0000 7f4f8bfff700 1 -- 172.21.15.23:0/3286150518 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f7c009550 con 0x7f4f8c0c4640 2023-03-27T10:02:05.870 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.867+0000 7f4f8bfff700 1 --2- 172.21.15.23:0/3286150518 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f4f8c0c4640 0x7f4f8c150150 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4f8400eba0 tx=0x7f4f8400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:05.870 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.867+0000 7f4f73fff700 1 -- 172.21.15.23:0/3286150518 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f84009a80 con 0x7f4f8c0c4640 2023-03-27T10:02:05.871 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.867+0000 7f4f73fff700 1 -- 172.21.15.23:0/3286150518 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f8400f040 con 0x7f4f8c0c4640 2023-03-27T10:02:05.871 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.867+0000 7f4f73fff700 1 -- 172.21.15.23:0/3286150518 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8400cc40 con 0x7f4f8c0c4640 2023-03-27T10:02:05.871 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.867+0000 7f4f91dff700 1 -- 172.21.15.23:0/3286150518 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f8c159610 con 0x7f4f8c0c4640 2023-03-27T10:02:05.871 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.867+0000 7f4f91dff700 1 -- 172.21.15.23:0/3286150518 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f8c159350 con 0x7f4f8c0c4640 2023-03-27T10:02:05.871 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.868+0000 7f4f91dff700 1 -- 172.21.15.23:0/3286150518 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f8c159f00 con 0x7f4f8c0c4640 2023-03-27T10:02:05.872 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.869+0000 7f4f73fff700 1 -- 172.21.15.23:0/3286150518 <== mon.0 v2:172.21.15.23:3300/0 4 ==== mgrmap(e 5) v1 ==== 85073+0+0 (secure 0 0 0) 0x7f4f84014480 con 0x7f4f8c0c4640 2023-03-27T10:02:05.872 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.870+0000 7f4f73fff700 1 --2- 172.21.15.23:0/3286150518 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f4f78066de0 0x7f4f78069290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:05.872 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.870+0000 7f4f73fff700 1 -- 172.21.15.23:0/3286150518 <== mon.0 v2:172.21.15.23:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 5188+0+0 (secure 0 0 0) 0x7f4f84084a90 con 0x7f4f8c0c4640 2023-03-27T10:02:05.872 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.870+0000 7f4f8b7fe700 1 --2- 172.21.15.23:0/3286150518 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f4f78066de0 0x7f4f78069290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:05.872 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.871+0000 7f4f8b7fe700 1 --2- 172.21.15.23:0/3286150518 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f4f78066de0 0x7f4f78069290 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c009c10 tx=0x7f4f7c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:05.877 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:05.875+0000 7f4f73fff700 1 -- 172.21.15.23:0/3286150518 <== mon.0 v2:172.21.15.23:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173280 (secure 0 0 0) 0x7f4f84052020 con 0x7f4f8c0c4640 2023-03-27T10:02:06.074 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.075+0000 7f4f91dff700 1 -- 172.21.15.23:0/3286150518 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4f8c0691b0 con 0x7f4f8c0c4640 2023-03-27T10:02:06.654 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.655+0000 7f4f73fff700 1 -- 172.21.15.23:0/3286150518 <== mon.0 v2:172.21.15.23:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v15) v1 ==== 173+0+0 (secure 0 0 0) 0x7f4f84055640 con 0x7f4f8c0c4640 2023-03-27T10:02:06.657 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.658+0000 7f4f91dff700 1 -- 172.21.15.23:0/3286150518 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f4f78066de0 msgr2=0x7f4f78069290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:06.658 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.658+0000 7f4f91dff700 1 --2- 172.21.15.23:0/3286150518 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f4f78066de0 0x7f4f78069290 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c009c10 tx=0x7f4f7c008030 comp rx=0 tx=0).stop 2023-03-27T10:02:06.658 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.658+0000 7f4f91dff700 1 -- 172.21.15.23:0/3286150518 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f4f8c0c4640 msgr2=0x7f4f8c150150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:06.658 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.658+0000 7f4f91dff700 1 --2- 172.21.15.23:0/3286150518 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f4f8c0c4640 0x7f4f8c150150 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4f8400eba0 tx=0x7f4f8400eeb0 comp rx=0 tx=0).stop 2023-03-27T10:02:06.658 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.658+0000 7f4f91dff700 1 -- 172.21.15.23:0/3286150518 shutdown_connections 2023-03-27T10:02:06.658 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.658+0000 7f4f91dff700 1 --2- 172.21.15.23:0/3286150518 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f4f8c0c4640 0x7f4f8c150150 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:06.658 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.658+0000 7f4f91dff700 1 --2- 172.21.15.23:0/3286150518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f8c06b0b0 0x7f4f8c1f2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:06.659 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.658+0000 7f4f91dff700 1 --2- 172.21.15.23:0/3286150518 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f4f78066de0 0x7f4f78069290 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:06.659 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.658+0000 7f4f91dff700 1 --2- 172.21.15.23:0/3286150518 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4f8c06b480 0x7f4f8c0b3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:06.659 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.658+0000 7f4f91dff700 1 -- 172.21.15.23:0/3286150518 >> 172.21.15.23:0/3286150518 conn(0x7f4f8c06beb0 msgr2=0x7f4f8c1528c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:06.659 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.658+0000 7f4f91dff700 1 -- 172.21.15.23:0/3286150518 shutdown_connections 2023-03-27T10:02:06.659 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.658+0000 7f4f91dff700 1 -- 172.21.15.23:0/3286150518 wait complete. 2023-03-27T10:02:06.660 INFO:teuthology.orchestra.run.smithi023.stderr:enabled application 'rbd' on pool 'rbd' 2023-03-27T10:02:06.678 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-03-27T10:02:06.679 INFO:tasks.ceph.mds.a:Restarting daemon 2023-03-27T10:02:06.679 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-03-27T10:02:06.683 INFO:tasks.ceph.mds.a:Started 2023-03-27T10:02:06.684 INFO:tasks.ceph.mds.c:Restarting daemon 2023-03-27T10:02:06.685 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-03-27T10:02:06.689 INFO:tasks.ceph.mds.c:Started 2023-03-27T10:02:06.689 INFO:tasks.ceph.mds.e:Restarting daemon 2023-03-27T10:02:06.689 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2023-03-27T10:02:06.693 INFO:tasks.ceph.mds.e:Started 2023-03-27T10:02:06.694 INFO:tasks.ceph.mds.b:Restarting daemon 2023-03-27T10:02:06.694 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-03-27T10:02:06.697 INFO:tasks.ceph.mds.b:Started 2023-03-27T10:02:06.697 INFO:tasks.ceph.mds.d:Restarting daemon 2023-03-27T10:02:06.697 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-03-27T10:02:06.699 INFO:tasks.ceph.mds.d:Started 2023-03-27T10:02:06.700 INFO:tasks.ceph.mds.f:Restarting daemon 2023-03-27T10:02:06.700 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2023-03-27T10:02:06.702 INFO:tasks.ceph.mds.f:Started 2023-03-27T10:02:06.702 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-03-27T10:02:06.703 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-27T10:02:06.791 INFO:tasks.ceph.mds.b.smithi112.stdout:starting mds.b at 2023-03-27T10:02:06.792 INFO:tasks.ceph.mds.d.smithi112.stdout:starting mds.d at 2023-03-27T10:02:06.793 INFO:tasks.ceph.mds.f.smithi112.stdout:starting mds.f at 2023-03-27T10:02:06.859 INFO:tasks.ceph.mds.a.smithi023.stdout:starting mds.a at 2023-03-27T10:02:06.899 INFO:tasks.ceph.mds.c.smithi023.stdout:starting mds.c at 2023-03-27T10:02:06.900 INFO:tasks.ceph.mds.e.smithi023.stdout:starting mds.e at 2023-03-27T10:02:06.955 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.956+0000 7f8ede1ff700 1 Processor -- start 2023-03-27T10:02:06.956 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.956+0000 7f8ede1ff700 1 -- start start 2023-03-27T10:02:06.957 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.957+0000 7f8ede1ff700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f8ed80c6990 0x7f8ed80c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:06.957 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.957+0000 7f8ede1ff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8ed806b440 0x7f8ed806b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:06.957 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.957+0000 7f8ede1ff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ed80ca730 con 0x7f8ed80c9520 2023-03-27T10:02:06.957 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.957+0000 7f8ede1ff700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ed80c72a0 con 0x7f8ed80c6990 2023-03-27T10:02:06.957 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.957+0000 7f8ede1ff700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ed80cf9f0 con 0x7f8ed806b440 2023-03-27T10:02:06.957 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.957+0000 7f8ed77fe700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f8ed80c6990 0x7f8ed80c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:06.958 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.957+0000 7f8ed77fe700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f8ed80c6990 0x7f8ed80c6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:57750/0 (socket says 172.21.15.23:57750) 2023-03-27T10:02:06.958 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.957+0000 7f8ed77fe700 1 -- 172.21.15.23:0/2449849991 learned_addr learned my addr 172.21.15.23:0/2449849991 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:06.958 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.957+0000 7f8ed6ffd700 1 --1- 172.21.15.23:0/2449849991 >> v1:172.21.15.112:6789/0 conn(0x7f8ed80c9520 0x7f8ed80c8820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.23:50140/0 (socket says 172.21.15.23:50140) 2023-03-27T10:02:06.958 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.957+0000 7f8ed7fff700 1 --2- 172.21.15.23:0/2449849991 >> v2:172.21.15.112:3300/0 conn(0x7f8ed806b440 0x7f8ed806b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:06.959 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.958+0000 7f8ed77fe700 1 -- 172.21.15.23:0/2449849991 >> v2:172.21.15.112:3300/0 conn(0x7f8ed806b440 msgr2=0x7f8ed806b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:02:06.959 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.958+0000 7f8ed77fe700 1 --2- 172.21.15.23:0/2449849991 >> v2:172.21.15.112:3300/0 conn(0x7f8ed806b440 0x7f8ed806b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:06.959 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.958+0000 7f8ed77fe700 1 -- 172.21.15.23:0/2449849991 >> v1:172.21.15.112:6789/0 conn(0x7f8ed80c9520 legacy=0x7f8ed80c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:06.960 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.958+0000 7f8ed67fc700 1 -- 172.21.15.23:0/2449849991 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 110395261 0 0) 0x7f8ed80ca730 con 0x7f8ed80c9520 2023-03-27T10:02:06.960 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.958+0000 7f8ed77fe700 1 -- 172.21.15.23:0/2449849991 --> v2:172.21.15.23:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed80c8f30 con 0x7f8ed80c6990 2023-03-27T10:02:06.960 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.958+0000 7f8ed77fe700 1 --2- 172.21.15.23:0/2449849991 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f8ed80c6990 0x7f8ed80c6d60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8ed806c7f0 tx=0x7f8ec8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52306c4809697fb7 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:06.960 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.958+0000 7f8ed67fc700 1 -- 172.21.15.23:0/2449849991 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8011830 con 0x7f8ed80c6990 2023-03-27T10:02:06.960 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.958+0000 7f8ed67fc700 1 -- 172.21.15.23:0/2449849991 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec8011990 con 0x7f8ed80c6990 2023-03-27T10:02:06.960 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.958+0000 7f8ed67fc700 1 -- 172.21.15.23:0/2449849991 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8011d20 con 0x7f8ed80c6990 2023-03-27T10:02:06.960 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.959+0000 7f8ede1ff700 1 -- 172.21.15.23:0/2449849991 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f8ed80c6990 msgr2=0x7f8ed80c6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:06.961 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.959+0000 7f8ede1ff700 1 --2- 172.21.15.23:0/2449849991 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f8ed80c6990 0x7f8ed80c6d60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8ed806c7f0 tx=0x7f8ec8009bd0 comp rx=0 tx=0).stop 2023-03-27T10:02:06.961 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.959+0000 7f8ede1ff700 1 -- 172.21.15.23:0/2449849991 shutdown_connections 2023-03-27T10:02:06.961 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.959+0000 7f8ede1ff700 1 --2- 172.21.15.23:0/2449849991 >> v2:172.21.15.112:3300/0 conn(0x7f8ed806b440 0x7f8ed806b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:06.961 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.959+0000 7f8ede1ff700 1 --2- 172.21.15.23:0/2449849991 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f8ed80c6990 0x7f8ed80c6d60 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:06.961 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.959+0000 7f8ede1ff700 1 -- 172.21.15.23:0/2449849991 >> 172.21.15.23:0/2449849991 conn(0x7f8ed8071920 msgr2=0x7f8ed8071cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:06.961 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.959+0000 7f8ede1ff700 1 -- 172.21.15.23:0/2449849991 shutdown_connections 2023-03-27T10:02:06.962 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.959+0000 7f8ede1ff700 1 -- 172.21.15.23:0/2449849991 wait complete. 2023-03-27T10:02:06.963 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.961+0000 7f8ede1ff700 1 Processor -- start 2023-03-27T10:02:06.964 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.961+0000 7f8ede1ff700 1 -- start start 2023-03-27T10:02:06.964 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.962+0000 7f8ede1ff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ed806b440 0x7f8ed81eec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:06.964 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.962+0000 7f8ede1ff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f8ed80c6990 0x7f8ed8155ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:06.964 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.962+0000 7f8ede1ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ed80c9520 0x7f8ed815b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:06.964 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.962+0000 7f8ede1ff700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed81e25e0 con 0x7f8ed80c6990 2023-03-27T10:02:06.964 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.962+0000 7f8ede1ff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed81e21a0 con 0x7f8ed806b440 2023-03-27T10:02:06.965 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.962+0000 7f8ede1ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed81ef170 con 0x7f8ed80c9520 2023-03-27T10:02:06.965 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.962+0000 7f8ed6ffd700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f8ed80c6990 0x7f8ed8155ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:06.965 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.962+0000 7f8ed77fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ed806b440 0x7f8ed81eec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:06.965 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.962+0000 7f8ed6ffd700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f8ed80c6990 0x7f8ed8155ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:57764/0 (socket says 172.21.15.23:57764) 2023-03-27T10:02:06.965 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.962+0000 7f8ed77fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ed806b440 0x7f8ed81eec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.23:52272/0 (socket says 172.21.15.23:52272) 2023-03-27T10:02:06.966 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.962+0000 7f8ed6ffd700 1 -- 172.21.15.23:0/703063806 learned_addr learned my addr 172.21.15.23:0/703063806 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:06.966 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.962+0000 7f8ed7fff700 1 --2- 172.21.15.23:0/703063806 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ed80c9520 0x7f8ed815b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:06.966 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.962+0000 7f8ed6ffd700 1 -- 172.21.15.23:0/703063806 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ed80c9520 msgr2=0x7f8ed815b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:06.966 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.962+0000 7f8ed6ffd700 1 --2- 172.21.15.23:0/703063806 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ed80c9520 0x7f8ed815b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:06.966 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.962+0000 7f8ed6ffd700 1 -- 172.21.15.23:0/703063806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ed806b440 msgr2=0x7f8ed81eec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:06.970 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.962+0000 7f8ed6ffd700 1 --2- 172.21.15.23:0/703063806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ed806b440 0x7f8ed81eec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:06.970 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.962+0000 7f8ed6ffd700 1 -- 172.21.15.23:0/703063806 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec8009550 con 0x7f8ed80c6990 2023-03-27T10:02:06.970 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.962+0000 7f8ed77fe700 1 --2- 172.21.15.23:0/703063806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ed806b440 0x7f8ed81eec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-27T10:02:06.970 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.963+0000 7f8ed6ffd700 1 --2- 172.21.15.23:0/703063806 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f8ed80c6990 0x7f8ed8155ce0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc00c580 tx=0x7f8ecc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:06.971 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.963+0000 7f8ebffff700 1 -- 172.21.15.23:0/703063806 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc004c80 con 0x7f8ed80c6990 2023-03-27T10:02:06.971 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.963+0000 7f8ebffff700 1 -- 172.21.15.23:0/703063806 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ecc00ce90 con 0x7f8ed80c6990 2023-03-27T10:02:06.971 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.963+0000 7f8ebffff700 1 -- 172.21.15.23:0/703063806 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc0070d0 con 0x7f8ed80c6990 2023-03-27T10:02:06.971 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.963+0000 7f8ede1ff700 1 -- 172.21.15.23:0/703063806 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ed8158ce0 con 0x7f8ed80c6990 2023-03-27T10:02:06.971 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.963+0000 7f8ede1ff700 1 -- 172.21.15.23:0/703063806 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ed8158a00 con 0x7f8ed80c6990 2023-03-27T10:02:06.971 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.964+0000 7f8ede1ff700 1 -- 172.21.15.23:0/703063806 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ed8158fc0 con 0x7f8ed80c6990 2023-03-27T10:02:06.972 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.966+0000 7f8ebffff700 1 -- 172.21.15.23:0/703063806 <== mon.0 v2:172.21.15.23:3300/0 4 ==== mgrmap(e 5) v1 ==== 85073+0+0 (secure 0 0 0) 0x7f8ecc01a020 con 0x7f8ed80c6990 2023-03-27T10:02:06.972 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.967+0000 7f8ebffff700 1 --2- 172.21.15.23:0/703063806 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f8ec4066de0 0x7f8ec4069290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:06.972 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.967+0000 7f8ebffff700 1 -- 172.21.15.23:0/703063806 <== mon.0 v2:172.21.15.23:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8ecc083980 con 0x7f8ed80c6990 2023-03-27T10:02:06.973 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.968+0000 7f8ed77fe700 1 --2- 172.21.15.23:0/703063806 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f8ec4066de0 0x7f8ec4069290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:06.973 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.968+0000 7f8ed77fe700 1 --2- 172.21.15.23:0/703063806 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f8ec4066de0 0x7f8ec4069290 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009c10 tx=0x7f8ec8008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:06.973 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:06.972+0000 7f8ebffff700 1 -- 172.21.15.23:0/703063806 <== mon.0 v2:172.21.15.23:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173280 (secure 0 0 0) 0x7f8ecc050f60 con 0x7f8ed80c6990 2023-03-27T10:02:07.193 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.193+0000 7f8ede1ff700 1 -- 172.21.15.23:0/703063806 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ed81e1f20 con 0x7f8ed80c6990 2023-03-27T10:02:07.193 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.194+0000 7f8ebffff700 1 -- 172.21.15.23:0/703063806 <== mon.0 v2:172.21.15.23:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+14268 (secure 0 0 0) 0x7f8ecc054580 con 0x7f8ed80c6990 2023-03-27T10:02:07.194 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:02:07.194 INFO:teuthology.orchestra.run.smithi023.stdout:{"epoch":15,"fsid":"1d8d6e6e-0d21-4f51-91ee-d9434d4fb750","created":"2023-03-27T10:01:53.805778+0000","modified":"2023-03-27T10:02:06.649924+0000","last_up_change":"2023-03-27T10:02:04.172266+0000","last_in_change":"2023-03-27T10:01:56.475091+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-27T10:02:05.288883+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-27T10:02:05.643442+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df086052-536b-4fb1-bcce-7aaf5a13911e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6800","nonce":96642},{"type":"v1","addr":"172.21.15.23:6801","nonce":96642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6802","nonce":96642},{"type":"v1","addr":"172.21.15.23:6803","nonce":96642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6806","nonce":96642},{"type":"v1","addr":"172.21.15.23:6807","nonce":96642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6804","nonce":96642},{"type":"v1","addr":"172.21.15.23:6805","nonce":96642}]},"public_addr":"172.21.15.23:6801/96642","cluster_addr":"172.21.15.23:6803/96642","heartbeat_back_addr":"172.21.15.23:6807/96642","heartbeat_front_addr":"172.21.15.23:6805/96642","state":["exists","up"]},{"osd":1,"uuid":"03138001-a3fd-43fc-9797-a47643d9a3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6808","nonce":96643},{"type":"v1","addr":"172.21.15.23:6809","nonce":96643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6810","nonce":96643},{"type":"v1","addr":"172.21.15.23:6811","nonce":96643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6814","nonce":96643},{"type":"v1","addr":"172.21.15.23:6815","nonce":96643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6812","nonce":96643},{"type":"v1","addr":"172.21.15.23:6813","nonce":96643}]},"public_addr":"172.21.15.23:6809/96643","cluster_addr":"172.21.15.23:6811/96643","heartbeat_back_addr":"172.21.15.23:6815/96643","heartbeat_front_addr":"172.21.15.23:6813/96643","state":["exists","up"]},{"osd":2,"uuid":"964dd926-1160-44ef-aa34-b18965dc05f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6817","nonce":96644},{"type":"v1","addr":"172.21.15.23:6819","nonce":96644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6820","nonce":96644},{"type":"v1","addr":"172.21.15.23:6822","nonce":96644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6827","nonce":96644},{"type":"v1","addr":"172.21.15.23:6829","nonce":96644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6824","nonce":96644},{"type":"v1","addr":"172.21.15.23:6825","nonce":96644}]},"public_addr":"172.21.15.23:6819/96644","cluster_addr":"172.21.15.23:6822/96644","heartbeat_back_addr":"172.21.15.23:6829/96644","heartbeat_front_addr":"172.21.15.23:6825/96644","state":["exists","up"]},{"osd":3,"uuid":"4ae5500f-cb24-44a8-a536-5c399c09e566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6816","nonce":96645},{"type":"v1","addr":"172.21.15.23:6818","nonce":96645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6821","nonce":96645},{"type":"v1","addr":"172.21.15.23:6823","nonce":96645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6830","nonce":96645},{"type":"v1","addr":"172.21.15.23:6831","nonce":96645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6826","nonce":96645},{"type":"v1","addr":"172.21.15.23:6828","nonce":96645}]},"public_addr":"172.21.15.23:6818/96645","cluster_addr":"172.21.15.23:6823/96645","heartbeat_back_addr":"172.21.15.23:6831/96645","heartbeat_front_addr":"172.21.15.23:6828/96645","state":["exists","up"]},{"osd":4,"uuid":"de228ab2-92b4-40bc-8dbd-8f2f4895dc22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":127986},{"type":"v1","addr":"172.21.15.112:6809","nonce":127986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":127986},{"type":"v1","addr":"172.21.15.112:6811","nonce":127986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6815","nonce":127986},{"type":"v1","addr":"172.21.15.112:6818","nonce":127986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":127986},{"type":"v1","addr":"172.21.15.112:6813","nonce":127986}]},"public_addr":"172.21.15.112:6809/127986","cluster_addr":"172.21.15.112:6811/127986","heartbeat_back_addr":"172.21.15.112:6818/127986","heartbeat_front_addr":"172.21.15.112:6813/127986","state":["exists","up"]},{"osd":5,"uuid":"6c839571-b490-4e66-92ac-0d7301c2e677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":127987},{"type":"v1","addr":"172.21.15.112:6801","nonce":127987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":127987},{"type":"v1","addr":"172.21.15.112:6803","nonce":127987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":127987},{"type":"v1","addr":"172.21.15.112:6807","nonce":127987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":127987},{"type":"v1","addr":"172.21.15.112:6805","nonce":127987}]},"public_addr":"172.21.15.112:6801/127987","cluster_addr":"172.21.15.112:6803/127987","heartbeat_back_addr":"172.21.15.112:6807/127987","heartbeat_front_addr":"172.21.15.112:6805/127987","state":["exists","up"]},{"osd":6,"uuid":"871b1a2b-f0ed-4faf-8f9a-a5a0973e11c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":127988},{"type":"v1","addr":"172.21.15.112:6816","nonce":127988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":127988},{"type":"v1","addr":"172.21.15.112:6819","nonce":127988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":127988},{"type":"v1","addr":"172.21.15.112:6823","nonce":127988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":127988},{"type":"v1","addr":"172.21.15.112:6821","nonce":127988}]},"public_addr":"172.21.15.112:6816/127988","cluster_addr":"172.21.15.112:6819/127988","heartbeat_back_addr":"172.21.15.112:6823/127988","heartbeat_front_addr":"172.21.15.112:6821/127988","state":["exists","up"]},{"osd":7,"uuid":"d8228353-34b5-4363-84d8-642512f5bc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":128009},{"type":"v1","addr":"172.21.15.112:6825","nonce":128009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":128009},{"type":"v1","addr":"172.21.15.112:6827","nonce":128009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":128009},{"type":"v1","addr":"172.21.15.112:6831","nonce":128009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":128009},{"type":"v1","addr":"172.21.15.112:6829","nonce":128009}]},"public_addr":"172.21.15.112:6825/128009","cluster_addr":"172.21.15.112:6827/128009","heartbeat_back_addr":"172.21.15.112:6831/128009","heartbeat_front_addr":"172.21.15.112:6829/128009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:02.132160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:01.882517+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:01.927673+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:01.867525+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:01.723580+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:01.730394+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:01.716687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:01.744322+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-27T10:02:07.196 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.197+0000 7f8ede1ff700 1 -- 172.21.15.23:0/703063806 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f8ec4066de0 msgr2=0x7f8ec4069290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:07.196 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.197+0000 7f8ede1ff700 1 --2- 172.21.15.23:0/703063806 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f8ec4066de0 0x7f8ec4069290 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009c10 tx=0x7f8ec8008030 comp rx=0 tx=0).stop 2023-03-27T10:02:07.197 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.197+0000 7f8ede1ff700 1 -- 172.21.15.23:0/703063806 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f8ed80c6990 msgr2=0x7f8ed8155ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:07.197 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.197+0000 7f8ede1ff700 1 --2- 172.21.15.23:0/703063806 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f8ed80c6990 0x7f8ed8155ce0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc00c580 tx=0x7f8ecc00c890 comp rx=0 tx=0).stop 2023-03-27T10:02:07.197 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.197+0000 7f8ede1ff700 1 -- 172.21.15.23:0/703063806 shutdown_connections 2023-03-27T10:02:07.197 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.197+0000 7f8ede1ff700 1 --2- 172.21.15.23:0/703063806 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f8ec4066de0 0x7f8ec4069290 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:07.197 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.197+0000 7f8ede1ff700 1 --2- 172.21.15.23:0/703063806 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ed80c9520 0x7f8ed815b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:07.197 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.197+0000 7f8ede1ff700 1 --2- 172.21.15.23:0/703063806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ed806b440 0x7f8ed81eec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:07.197 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.197+0000 7f8ede1ff700 1 --2- 172.21.15.23:0/703063806 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f8ed80c6990 0x7f8ed8155ce0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:07.198 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.197+0000 7f8ede1ff700 1 -- 172.21.15.23:0/703063806 >> 172.21.15.23:0/703063806 conn(0x7f8ed8071920 msgr2=0x7f8ed80ce2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:07.198 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.197+0000 7f8ede1ff700 1 -- 172.21.15.23:0/703063806 shutdown_connections 2023-03-27T10:02:07.198 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.197+0000 7f8ede1ff700 1 -- 172.21.15.23:0/703063806 wait complete. 2023-03-27T10:02:07.214 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-03-27T10:02:05.288883+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-03-27T10:02:05.643442+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-03-27T10:02:07.214 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-27T10:02:07.408 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.408+0000 7f67a39ff700 1 Processor -- start 2023-03-27T10:02:07.408 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.409+0000 7f67a39ff700 1 -- start start 2023-03-27T10:02:07.409 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.410+0000 7f67a39ff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f679c0c6990 0x7f679c0c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:07.410 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.410+0000 7f67a39ff700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f679c0c9520 0x7f679c0c8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:07.410 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.410+0000 7f67a39ff700 1 -- --> v1:172.21.15.23:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f679c0ca730 con 0x7f679c06b440 2023-03-27T10:02:07.410 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.410+0000 7f67a39ff700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f679c0c72a0 con 0x7f679c0c9520 2023-03-27T10:02:07.410 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.410+0000 7f67a39ff700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f679c0cf9f0 con 0x7f679c0c6990 2023-03-27T10:02:07.410 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.410+0000 7f67a0f9a700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f679c0c9520 0x7f679c0c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:07.411 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.410+0000 7f67a179b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f679c0c6990 0x7f679c0c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:07.411 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.410+0000 7f67a179b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f679c0c6990 0x7f679c0c6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.23:33746/0 (socket says 172.21.15.23:33746) 2023-03-27T10:02:07.411 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.410+0000 7f67a0f9a700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f679c0c9520 0x7f679c0c8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:48596/0 (socket says 172.21.15.23:48596) 2023-03-27T10:02:07.411 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.410+0000 7f67a179b700 1 -- 172.21.15.23:0/336595996 learned_addr learned my addr 172.21.15.23:0/336595996 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:07.412 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.411+0000 7f67a179b700 1 -- 172.21.15.23:0/336595996 >> v2:172.21.15.23:3300/0 conn(0x7f679c0c9520 msgr2=0x7f679c0c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:02:07.412 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.411+0000 7f67a179b700 1 --2- 172.21.15.23:0/336595996 >> v2:172.21.15.23:3300/0 conn(0x7f679c0c9520 0x7f679c0c8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:07.412 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.411+0000 7f67a179b700 1 -- 172.21.15.23:0/336595996 >> v1:172.21.15.23:6789/0 conn(0x7f679c06b440 legacy=0x7f679c06b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:07.413 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.411+0000 7f67a179b700 1 -- 172.21.15.23:0/336595996 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f679c158a00 con 0x7f679c0c6990 2023-03-27T10:02:07.413 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.411+0000 7f67a179b700 1 --2- 172.21.15.23:0/336595996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f679c0c6990 0x7f679c0c6d60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f679c072420 tx=0x7f6798009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85d6c70b97f2c0a7 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:07.413 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.411+0000 7f6793fff700 1 -- 172.21.15.23:0/336595996 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6798011830 con 0x7f679c0c6990 2023-03-27T10:02:07.413 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.411+0000 7f6793fff700 1 -- 172.21.15.23:0/336595996 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6798011990 con 0x7f679c0c6990 2023-03-27T10:02:07.413 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.411+0000 7f6793fff700 1 -- 172.21.15.23:0/336595996 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6798011d00 con 0x7f679c0c6990 2023-03-27T10:02:07.413 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.412+0000 7f67a39ff700 1 -- 172.21.15.23:0/336595996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f679c0c6990 msgr2=0x7f679c0c6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:07.414 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.412+0000 7f67a39ff700 1 --2- 172.21.15.23:0/336595996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f679c0c6990 0x7f679c0c6d60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f679c072420 tx=0x7f6798009bd0 comp rx=0 tx=0).stop 2023-03-27T10:02:07.414 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.412+0000 7f67a39ff700 1 -- 172.21.15.23:0/336595996 shutdown_connections 2023-03-27T10:02:07.414 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.412+0000 7f67a39ff700 1 --2- 172.21.15.23:0/336595996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f679c0c6990 0x7f679c0c6d60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:07.414 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.412+0000 7f67a39ff700 1 --2- 172.21.15.23:0/336595996 >> v2:172.21.15.23:3300/0 conn(0x7f679c0c9520 0x7f679c0c8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:07.414 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.412+0000 7f67a39ff700 1 -- 172.21.15.23:0/336595996 >> 172.21.15.23:0/336595996 conn(0x7f679c071920 msgr2=0x7f679c071cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:07.415 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.412+0000 7f67a39ff700 1 -- 172.21.15.23:0/336595996 shutdown_connections 2023-03-27T10:02:07.415 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.412+0000 7f67a39ff700 1 -- 172.21.15.23:0/336595996 wait complete. 2023-03-27T10:02:07.417 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.414+0000 7f67a39ff700 1 Processor -- start 2023-03-27T10:02:07.417 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.414+0000 7f67a39ff700 1 -- start start 2023-03-27T10:02:07.417 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.415+0000 7f67a39ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f679c06b440 0x7f679c151740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:07.417 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.415+0000 7f67a39ff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f679c0c6990 0x7f679c15a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:07.418 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.415+0000 7f67a39ff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f679c0c9520 0x7f679c15b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:07.418 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.415+0000 7f67a39ff700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679c151c80 con 0x7f679c0c6990 2023-03-27T10:02:07.418 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.415+0000 7f67a39ff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679c14cea0 con 0x7f679c0c9520 2023-03-27T10:02:07.418 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.415+0000 7f67a39ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f679c15aba0 con 0x7f679c06b440 2023-03-27T10:02:07.418 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.415+0000 7f67a0f9a700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f679c0c6990 0x7f679c15a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:07.418 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.415+0000 7f67a1f9c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f679c0c9520 0x7f679c15b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:07.418 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.415+0000 7f67a1f9c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f679c0c9520 0x7f679c15b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.23:33752/0 (socket says 172.21.15.23:33752) 2023-03-27T10:02:07.418 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.415+0000 7f67a0f9a700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f679c0c6990 0x7f679c15a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:48600/0 (socket says 172.21.15.23:48600) 2023-03-27T10:02:07.419 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.415+0000 7f67a1f9c700 1 -- 172.21.15.23:0/3212479795 learned_addr learned my addr 172.21.15.23:0/3212479795 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:07.419 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.415+0000 7f67a179b700 1 --2- 172.21.15.23:0/3212479795 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f679c06b440 0x7f679c151740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:07.419 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.415+0000 7f67a0f9a700 1 -- 172.21.15.23:0/3212479795 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f679c06b440 msgr2=0x7f679c151740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:07.419 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.415+0000 7f67a0f9a700 1 --2- 172.21.15.23:0/3212479795 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f679c06b440 0x7f679c151740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:07.419 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.415+0000 7f67a0f9a700 1 -- 172.21.15.23:0/3212479795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f679c0c9520 msgr2=0x7f679c15b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:07.419 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.415+0000 7f67a0f9a700 1 --2- 172.21.15.23:0/3212479795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f679c0c9520 0x7f679c15b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:07.419 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.416+0000 7f67a0f9a700 1 -- 172.21.15.23:0/3212479795 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6798009550 con 0x7f679c0c6990 2023-03-27T10:02:07.420 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.416+0000 7f67a0f9a700 1 --2- 172.21.15.23:0/3212479795 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f679c0c6990 0x7f679c15a280 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f678c00e870 tx=0x7f678c00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:07.420 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.416+0000 7f6791ffb700 1 -- 172.21.15.23:0/3212479795 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c013680 con 0x7f679c0c6990 2023-03-27T10:02:07.420 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.416+0000 7f6791ffb700 1 -- 172.21.15.23:0/3212479795 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678c0137e0 con 0x7f679c0c6990 2023-03-27T10:02:07.420 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.416+0000 7f67a39ff700 1 -- 172.21.15.23:0/3212479795 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f679c158fc0 con 0x7f679c0c6990 2023-03-27T10:02:07.420 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.416+0000 7f6791ffb700 1 -- 172.21.15.23:0/3212479795 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c013b90 con 0x7f679c0c6990 2023-03-27T10:02:07.420 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.416+0000 7f67a39ff700 1 -- 172.21.15.23:0/3212479795 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f679c158ce0 con 0x7f679c0c6990 2023-03-27T10:02:07.420 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.417+0000 7f67a39ff700 1 -- 172.21.15.23:0/3212479795 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f679c1592a0 con 0x7f679c0c6990 2023-03-27T10:02:07.421 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.418+0000 7f6791ffb700 1 -- 172.21.15.23:0/3212479795 <== mon.0 v2:172.21.15.23:3300/0 4 ==== mgrmap(e 5) v1 ==== 85073+0+0 (secure 0 0 0) 0x7f678c023070 con 0x7f679c0c6990 2023-03-27T10:02:07.421 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.419+0000 7f6791ffb700 1 --2- 172.21.15.23:0/3212479795 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f6780066de0 0x7f6780069290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:07.422 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.419+0000 7f6791ffb700 1 -- 172.21.15.23:0/3212479795 <== mon.0 v2:172.21.15.23:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f678c083ee0 con 0x7f679c0c6990 2023-03-27T10:02:07.422 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.419+0000 7f67a179b700 1 --2- 172.21.15.23:0/3212479795 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f6780066de0 0x7f6780069290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:07.422 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.420+0000 7f67a179b700 1 --2- 172.21.15.23:0/3212479795 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f6780066de0 0x7f6780069290 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6798009c10 tx=0x7f6798008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:07.422 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.423+0000 7f6791ffb700 1 -- 172.21.15.23:0/3212479795 <== mon.0 v2:172.21.15.23:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173280 (secure 0 0 0) 0x7f678c0514c0 con 0x7f679c0c6990 2023-03-27T10:02:07.628 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.628+0000 7f67a39ff700 1 -- 172.21.15.23:0/3212479795 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f679c069220 con 0x7f679c0c6990 2023-03-27T10:02:07.628 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.628+0000 7f6791ffb700 1 -- 172.21.15.23:0/3212479795 <== mon.0 v2:172.21.15.23:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v15) v1 ==== 93+0+10 (secure 0 0 0) 0x7f678c054ae0 con 0x7f679c0c6990 2023-03-27T10:02:07.628 INFO:teuthology.orchestra.run.smithi023.stdout:pg_num: 1 2023-03-27T10:02:07.630 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.631+0000 7f67a39ff700 1 -- 172.21.15.23:0/3212479795 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f6780066de0 msgr2=0x7f6780069290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:07.630 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.631+0000 7f67a39ff700 1 --2- 172.21.15.23:0/3212479795 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f6780066de0 0x7f6780069290 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6798009c10 tx=0x7f6798008030 comp rx=0 tx=0).stop 2023-03-27T10:02:07.630 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.631+0000 7f67a39ff700 1 -- 172.21.15.23:0/3212479795 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f679c0c6990 msgr2=0x7f679c15a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:07.631 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.631+0000 7f67a39ff700 1 --2- 172.21.15.23:0/3212479795 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f679c0c6990 0x7f679c15a280 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f678c00e870 tx=0x7f678c00eb80 comp rx=0 tx=0).stop 2023-03-27T10:02:07.631 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.631+0000 7f67a39ff700 1 -- 172.21.15.23:0/3212479795 shutdown_connections 2023-03-27T10:02:07.631 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.631+0000 7f67a39ff700 1 --2- 172.21.15.23:0/3212479795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f679c0c9520 0x7f679c15b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:07.631 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.631+0000 7f67a39ff700 1 --2- 172.21.15.23:0/3212479795 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f6780066de0 0x7f6780069290 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:07.631 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.631+0000 7f67a39ff700 1 --2- 172.21.15.23:0/3212479795 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f679c06b440 0x7f679c151740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:07.631 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.631+0000 7f67a39ff700 1 --2- 172.21.15.23:0/3212479795 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f679c0c6990 0x7f679c15a280 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:07.631 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.631+0000 7f67a39ff700 1 -- 172.21.15.23:0/3212479795 >> 172.21.15.23:0/3212479795 conn(0x7f679c071920 msgr2=0x7f679c0c87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:07.632 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.631+0000 7f67a39ff700 1 -- 172.21.15.23:0/3212479795 shutdown_connections 2023-03-27T10:02:07.632 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.631+0000 7f67a39ff700 1 -- 172.21.15.23:0/3212479795 wait complete. 2023-03-27T10:02:07.647 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-27T10:02:07.842 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.843+0000 7f81b73ff700 1 Processor -- start 2023-03-27T10:02:07.843 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.843+0000 7f81b73ff700 1 -- start start 2023-03-27T10:02:07.844 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.844+0000 7f81b73ff700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f81b00c9520 0x7f81b00c8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:07.844 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.844+0000 7f81b73ff700 1 -- --> v1:172.21.15.23:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81b00ca730 con 0x7f81b006b440 2023-03-27T10:02:07.844 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.844+0000 7f81b73ff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81b00cf9f0 con 0x7f81b00c6990 2023-03-27T10:02:07.844 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.844+0000 7f81b73ff700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81b0158a00 con 0x7f81b00c9520 2023-03-27T10:02:07.844 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.844+0000 7f81b499a700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f81b00c9520 0x7f81b00c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:07.844 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.844+0000 7f81b519b700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f81b00c6990 0x7f81b00c6d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.23:57220/0 (socket says 172.21.15.23:57220) 2023-03-27T10:02:07.845 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.844+0000 7f81b519b700 1 -- 172.21.15.23:0/1990865992 learned_addr learned my addr 172.21.15.23:0/1990865992 (peer_addr_for_me v1:172.21.15.23:0/0) 2023-03-27T10:02:07.846 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.845+0000 7f81b499a700 1 -- 172.21.15.23:0/1990865992 >> v1:172.21.15.112:6789/0 conn(0x7f81b00c6990 legacy=0x7f81b00c6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:07.846 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.845+0000 7f81b499a700 1 -- 172.21.15.23:0/1990865992 >> v1:172.21.15.23:6789/0 conn(0x7f81b006b440 legacy=0x7f81b006b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:07.846 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.845+0000 7f81a7fff700 1 -- 172.21.15.23:0/1990865992 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1799021386 0 0) 0x7f81b00cf9f0 con 0x7f81b00c6990 2023-03-27T10:02:07.846 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.845+0000 7f81b499a700 1 -- 172.21.15.23:0/1990865992 --> v2:172.21.15.23:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b0158ce0 con 0x7f81b00c9520 2023-03-27T10:02:07.846 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.845+0000 7f81b499a700 1 --2- 172.21.15.23:0/1990865992 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f81b00c9520 0x7f81b00c8820 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f81a0009840 tx=0x7f81a0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87f29c69722d89fd server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:07.847 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.845+0000 7f81a7fff700 1 -- 172.21.15.23:0/1990865992 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a0004230 con 0x7f81b00c9520 2023-03-27T10:02:07.847 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.845+0000 7f81a7fff700 1 -- 172.21.15.23:0/1990865992 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a0004390 con 0x7f81b00c9520 2023-03-27T10:02:07.847 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.845+0000 7f81a7fff700 1 -- 172.21.15.23:0/1990865992 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a00036a0 con 0x7f81b00c9520 2023-03-27T10:02:07.847 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.846+0000 7f81b73ff700 1 -- 172.21.15.23:0/1990865992 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f81b00c9520 msgr2=0x7f81b00c8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:07.847 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.846+0000 7f81b73ff700 1 --2- 172.21.15.23:0/1990865992 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f81b00c9520 0x7f81b00c8820 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f81a0009840 tx=0x7f81a0009b50 comp rx=0 tx=0).stop 2023-03-27T10:02:07.847 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.846+0000 7f81b73ff700 1 -- 172.21.15.23:0/1990865992 shutdown_connections 2023-03-27T10:02:07.847 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.846+0000 7f81b73ff700 1 --2- 172.21.15.23:0/1990865992 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f81b00c9520 0x7f81b00c8820 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:07.848 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.846+0000 7f81b73ff700 1 -- 172.21.15.23:0/1990865992 >> 172.21.15.23:0/1990865992 conn(0x7f81b0071920 msgr2=0x7f81b0071cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:07.848 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.846+0000 7f81b73ff700 1 -- 172.21.15.23:0/1990865992 shutdown_connections 2023-03-27T10:02:07.848 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.846+0000 7f81b73ff700 1 -- 172.21.15.23:0/1990865992 wait complete. 2023-03-27T10:02:07.848 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.849+0000 7f81b73ff700 1 Processor -- start 2023-03-27T10:02:07.849 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.849+0000 7f81b73ff700 1 -- start start 2023-03-27T10:02:07.850 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.850+0000 7f81b73ff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f81b006b440 0x7f81b0151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:07.850 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.850+0000 7f81b73ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81b00c6990 0x7f81b014f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:07.850 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.850+0000 7f81b73ff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81b00c9520 0x7f81b014fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:07.850 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.850+0000 7f81b73ff700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b0155490 con 0x7f81b006b440 2023-03-27T10:02:07.851 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.850+0000 7f81b73ff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b014f240 con 0x7f81b00c9520 2023-03-27T10:02:07.851 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.850+0000 7f81b73ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81b014ec30 con 0x7f81b00c6990 2023-03-27T10:02:07.851 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.851+0000 7f81b499a700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81b00c6990 0x7f81b014f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:07.851 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.851+0000 7f81b599c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81b00c9520 0x7f81b014fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:07.851 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.851+0000 7f81b499a700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81b00c6990 0x7f81b014f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.23:48970/0 (socket says 172.21.15.23:48970) 2023-03-27T10:02:07.851 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.851+0000 7f81b499a700 1 -- 172.21.15.23:0/4291133207 learned_addr learned my addr 172.21.15.23:0/4291133207 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:07.852 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.851+0000 7f81b499a700 1 -- 172.21.15.23:0/4291133207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81b00c9520 msgr2=0x7f81b014fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:07.853 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.851+0000 7f81b499a700 1 --2- 172.21.15.23:0/4291133207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81b00c9520 0x7f81b014fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:07.853 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.851+0000 7f81b499a700 1 -- 172.21.15.23:0/4291133207 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f81b006b440 msgr2=0x7f81b0151020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:07.853 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.851+0000 7f81b499a700 1 --2- 172.21.15.23:0/4291133207 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f81b006b440 0x7f81b0151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:07.853 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.851+0000 7f81b499a700 1 -- 172.21.15.23:0/4291133207 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a00094a0 con 0x7f81b00c6990 2023-03-27T10:02:07.853 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.851+0000 7f81b599c700 1 --2- 172.21.15.23:0/4291133207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81b00c9520 0x7f81b014fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-27T10:02:07.853 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.851+0000 7f81b499a700 1 --2- 172.21.15.23:0/4291133207 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81b00c6990 0x7f81b014f6d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f81a00090b0 tx=0x7f81a0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:07.854 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.852+0000 7f81a5ffb700 1 -- 172.21.15.23:0/4291133207 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a0003dc0 con 0x7f81b00c6990 2023-03-27T10:02:07.854 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.852+0000 7f81a5ffb700 1 -- 172.21.15.23:0/4291133207 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a001f070 con 0x7f81b00c6990 2023-03-27T10:02:07.854 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.852+0000 7f81a5ffb700 1 -- 172.21.15.23:0/4291133207 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a0011830 con 0x7f81b00c6990 2023-03-27T10:02:07.854 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.852+0000 7f81b73ff700 1 -- 172.21.15.23:0/4291133207 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81b01592a0 con 0x7f81b00c6990 2023-03-27T10:02:07.854 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.852+0000 7f81b73ff700 1 -- 172.21.15.23:0/4291133207 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81b0158fc0 con 0x7f81b00c6990 2023-03-27T10:02:07.854 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.853+0000 7f81b73ff700 1 -- 172.21.15.23:0/4291133207 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81b0159580 con 0x7f81b00c6990 2023-03-27T10:02:07.855 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.854+0000 7f81a5ffb700 1 -- 172.21.15.23:0/4291133207 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) v1 ==== 85073+0+0 (secure 0 0 0) 0x7f81a002a020 con 0x7f81b00c6990 2023-03-27T10:02:07.855 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.855+0000 7f81a5ffb700 1 --2- 172.21.15.23:0/4291133207 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f8194066e10 0x7f81940692c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:07.856 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.855+0000 7f81a5ffb700 1 -- 172.21.15.23:0/4291133207 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f81a0085480 con 0x7f81b00c6990 2023-03-27T10:02:07.856 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.855+0000 7f81b519b700 1 --2- 172.21.15.23:0/4291133207 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f8194066e10 0x7f81940692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:07.856 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.855+0000 7f81b519b700 1 --2- 172.21.15.23:0/4291133207 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f8194066e10 0x7f81940692c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f81ac0011e0 tx=0x7f81ac0079b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:07.858 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:07.859+0000 7f81a5ffb700 1 -- 172.21.15.23:0/4291133207 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173280 (secure 0 0 0) 0x7f81a0052b50 con 0x7f81b00c6990 2023-03-27T10:02:08.062 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.063+0000 7f81b73ff700 1 -- 172.21.15.23:0/4291133207 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f81b0159580 con 0x7f81b00c6990 2023-03-27T10:02:08.065 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.065+0000 7f81a5ffb700 1 -- 172.21.15.23:0/4291133207 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v16) v1 ==== 92+0+10 (secure 0 0 0) 0x7f81a0056170 con 0x7f81b00c6990 2023-03-27T10:02:08.065 INFO:teuthology.orchestra.run.smithi023.stdout:pg_num: 8 2023-03-27T10:02:08.068 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.068+0000 7f81b73ff700 1 -- 172.21.15.23:0/4291133207 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f8194066e10 msgr2=0x7f81940692c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:08.068 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.068+0000 7f81b73ff700 1 --2- 172.21.15.23:0/4291133207 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f8194066e10 0x7f81940692c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f81ac0011e0 tx=0x7f81ac0079b0 comp rx=0 tx=0).stop 2023-03-27T10:02:08.068 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.068+0000 7f81b73ff700 1 -- 172.21.15.23:0/4291133207 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81b00c6990 msgr2=0x7f81b014f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:08.069 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.068+0000 7f81b73ff700 1 --2- 172.21.15.23:0/4291133207 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81b00c6990 0x7f81b014f6d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f81a00090b0 tx=0x7f81a0004750 comp rx=0 tx=0).stop 2023-03-27T10:02:08.069 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.068+0000 7f81b73ff700 1 -- 172.21.15.23:0/4291133207 shutdown_connections 2023-03-27T10:02:08.069 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.068+0000 7f81b73ff700 1 --2- 172.21.15.23:0/4291133207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81b00c9520 0x7f81b014fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:08.070 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.068+0000 7f81b73ff700 1 --2- 172.21.15.23:0/4291133207 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f81b006b440 0x7f81b0151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:08.070 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.068+0000 7f81b73ff700 1 --2- 172.21.15.23:0/4291133207 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f8194066e10 0x7f81940692c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:08.071 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.068+0000 7f81b73ff700 1 --2- 172.21.15.23:0/4291133207 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81b00c6990 0x7f81b014f6d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:08.071 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.068+0000 7f81b73ff700 1 -- 172.21.15.23:0/4291133207 >> 172.21.15.23:0/4291133207 conn(0x7f81b0071920 msgr2=0x7f81b0071cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:08.071 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.068+0000 7f81b73ff700 1 -- 172.21.15.23:0/4291133207 shutdown_connections 2023-03-27T10:02:08.072 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.068+0000 7f81b73ff700 1 -- 172.21.15.23:0/4291133207 wait complete. 2023-03-27T10:02:08.086 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-03-27T10:02:08.276 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.277+0000 7f941b9ff700 1 Processor -- start 2023-03-27T10:02:08.277 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.277+0000 7f941b9ff700 1 -- start start 2023-03-27T10:02:08.284 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.278+0000 7f941b9ff700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f941414f840 0x7f94140c3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:08.284 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.278+0000 7f941b9ff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f94140c1c10 0x7f94140c39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:08.284 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.278+0000 7f941b9ff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94140cb3b0 con 0x7f94140c6430 2023-03-27T10:02:08.285 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.278+0000 7f941b9ff700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9414070630 con 0x7f941414f840 2023-03-27T10:02:08.285 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.278+0000 7f941b9ff700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94140708d0 con 0x7f94140c1c10 2023-03-27T10:02:08.285 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.278+0000 7f941979b700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f941414f840 0x7f94140c3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:08.285 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.279+0000 7f941979b700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f941414f840 0x7f94140c3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:48642/0 (socket says 172.21.15.23:48642) 2023-03-27T10:02:08.285 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.279+0000 7f941979b700 1 -- 172.21.15.23:0/4149430334 learned_addr learned my addr 172.21.15.23:0/4149430334 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:08.285 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.279+0000 7f9418f9a700 1 --2- 172.21.15.23:0/4149430334 >> v2:172.21.15.112:3300/0 conn(0x7f94140c1c10 0x7f94140c39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:08.285 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.279+0000 7f941979b700 1 -- 172.21.15.23:0/4149430334 >> v2:172.21.15.112:3300/0 conn(0x7f94140c1c10 msgr2=0x7f94140c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:02:08.286 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.279+0000 7f941979b700 1 --2- 172.21.15.23:0/4149430334 >> v2:172.21.15.112:3300/0 conn(0x7f94140c1c10 0x7f94140c39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:08.286 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.279+0000 7f941979b700 1 -- 172.21.15.23:0/4149430334 >> v1:172.21.15.112:6789/0 conn(0x7f94140c6430 legacy=0x7f9414155730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:08.286 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.279+0000 7f941979b700 1 -- 172.21.15.23:0/4149430334 --> v2:172.21.15.23:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94140ca7a0 con 0x7f941414f840 2023-03-27T10:02:08.286 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.279+0000 7f941979b700 1 --2- 172.21.15.23:0/4149430334 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f941414f840 0x7f94140c3460 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f941406f5c0 tx=0x7f9410009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2b8f4032e57cc90 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:08.286 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.279+0000 7f940bfff700 1 -- 172.21.15.23:0/4149430334 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9410011830 con 0x7f941414f840 2023-03-27T10:02:08.286 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.280+0000 7f940bfff700 1 -- 172.21.15.23:0/4149430334 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9410011990 con 0x7f941414f840 2023-03-27T10:02:08.286 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.280+0000 7f940bfff700 1 -- 172.21.15.23:0/4149430334 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9410011d20 con 0x7f941414f840 2023-03-27T10:02:08.287 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.280+0000 7f941b9ff700 1 -- 172.21.15.23:0/4149430334 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f941414f840 msgr2=0x7f94140c3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:08.287 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.280+0000 7f941b9ff700 1 --2- 172.21.15.23:0/4149430334 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f941414f840 0x7f94140c3460 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f941406f5c0 tx=0x7f9410009bd0 comp rx=0 tx=0).stop 2023-03-27T10:02:08.287 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.280+0000 7f941b9ff700 1 -- 172.21.15.23:0/4149430334 shutdown_connections 2023-03-27T10:02:08.287 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.280+0000 7f941b9ff700 1 --2- 172.21.15.23:0/4149430334 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f941414f840 0x7f94140c3460 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:08.287 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.280+0000 7f941b9ff700 1 --2- 172.21.15.23:0/4149430334 >> v2:172.21.15.112:3300/0 conn(0x7f94140c1c10 0x7f94140c39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:08.287 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.280+0000 7f941b9ff700 1 -- 172.21.15.23:0/4149430334 >> 172.21.15.23:0/4149430334 conn(0x7f941406a1f0 msgr2=0x7f941406a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:08.287 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.280+0000 7f941b9ff700 1 -- 172.21.15.23:0/4149430334 shutdown_connections 2023-03-27T10:02:08.287 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.280+0000 7f941b9ff700 1 -- 172.21.15.23:0/4149430334 wait complete. 2023-03-27T10:02:08.288 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.282+0000 7f941b9ff700 1 Processor -- start 2023-03-27T10:02:08.288 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.282+0000 7f941b9ff700 1 -- start start 2023-03-27T10:02:08.288 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.283+0000 7f941b9ff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94140c1c10 0x7f94141f1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:08.288 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.283+0000 7f941b9ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f94140c6430 0x7f94140c43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:08.288 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.283+0000 7f941b9ff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f941414f840 0x7f9414144270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:08.288 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.283+0000 7f941b9ff700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9414155340 con 0x7f941414f840 2023-03-27T10:02:08.288 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.283+0000 7f941b9ff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9414156b80 con 0x7f94140c1c10 2023-03-27T10:02:08.289 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.283+0000 7f941b9ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9414158890 con 0x7f94140c6430 2023-03-27T10:02:08.289 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.283+0000 7f941979b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94140c1c10 0x7f94141f1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:08.289 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.283+0000 7f941979b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94140c1c10 0x7f94141f1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.23:33762/0 (socket says 172.21.15.23:33762) 2023-03-27T10:02:08.289 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.283+0000 7f941979b700 1 -- 172.21.15.23:0/603728337 learned_addr learned my addr 172.21.15.23:0/603728337 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:08.289 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.283+0000 7f9418f9a700 1 --2- 172.21.15.23:0/603728337 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f94140c6430 0x7f94140c43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:08.289 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.283+0000 7f9419f9c700 1 --2- 172.21.15.23:0/603728337 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f941414f840 0x7f9414144270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:08.289 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.283+0000 7f941979b700 1 -- 172.21.15.23:0/603728337 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f94140c6430 msgr2=0x7f94140c43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:08.289 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.283+0000 7f941979b700 1 --2- 172.21.15.23:0/603728337 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f94140c6430 0x7f94140c43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:08.290 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.283+0000 7f941979b700 1 -- 172.21.15.23:0/603728337 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f941414f840 msgr2=0x7f9414144270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:08.290 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.283+0000 7f941979b700 1 --2- 172.21.15.23:0/603728337 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f941414f840 0x7f9414144270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:08.290 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.283+0000 7f941979b700 1 -- 172.21.15.23:0/603728337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9410009550 con 0x7f94140c1c10 2023-03-27T10:02:08.290 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.284+0000 7f941979b700 1 --2- 172.21.15.23:0/603728337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94140c1c10 0x7f94141f1510 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9410009c10 tx=0x7f9410004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:08.290 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.284+0000 7f9409ffb700 1 -- 172.21.15.23:0/603728337 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9410004510 con 0x7f94140c1c10 2023-03-27T10:02:08.290 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.284+0000 7f9409ffb700 1 -- 172.21.15.23:0/603728337 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f941001e070 con 0x7f94140c1c10 2023-03-27T10:02:08.290 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.284+0000 7f9409ffb700 1 -- 172.21.15.23:0/603728337 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94100038e0 con 0x7f94140c1c10 2023-03-27T10:02:08.291 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.284+0000 7f941b9ff700 1 -- 172.21.15.23:0/603728337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94140cad60 con 0x7f94140c1c10 2023-03-27T10:02:08.291 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.284+0000 7f941b9ff700 1 -- 172.21.15.23:0/603728337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94140caa80 con 0x7f94140c1c10 2023-03-27T10:02:08.292 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.285+0000 7f941b9ff700 1 -- 172.21.15.23:0/603728337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94140cb040 con 0x7f94140c1c10 2023-03-27T10:02:08.292 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.286+0000 7f9409ffb700 1 -- 172.21.15.23:0/603728337 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 85113+0+0 (secure 0 0 0) 0x7f941000f050 con 0x7f94140c1c10 2023-03-27T10:02:08.292 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.287+0000 7f9409ffb700 1 --2- 172.21.15.23:0/603728337 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f93f8066e30 0x7f93f80692e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:08.292 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.287+0000 7f9409ffb700 1 -- 172.21.15.23:0/603728337 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9410084f50 con 0x7f94140c1c10 2023-03-27T10:02:08.292 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.287+0000 7f9418f9a700 1 --2- 172.21.15.23:0/603728337 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f93f8066e30 0x7f93f80692e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:08.292 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.287+0000 7f9418f9a700 1 --2- 172.21.15.23:0/603728337 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f93f8066e30 0x7f93f80692e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f941414aa60 tx=0x7f9404009210 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:08.293 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.291+0000 7f9409ffb700 1 -- 172.21.15.23:0/603728337 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173280 (secure 0 0 0) 0x7f94100524e0 con 0x7f94140c1c10 2023-03-27T10:02:08.536 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.537+0000 7f941b9ff700 1 -- 172.21.15.23:0/603728337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9414069220 con 0x7f94140c1c10 2023-03-27T10:02:08.537 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.538+0000 7f9409ffb700 1 -- 172.21.15.23:0/603728337 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+5036 (secure 0 0 0) 0x7f9410055b00 con 0x7f94140c1c10 2023-03-27T10:02:08.537 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:02:08.537 INFO:teuthology.orchestra.run.smithi023.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.23:6836/4002126418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6834","nonce":4002126418},{"type":"v1","addr":"172.21.15.23:6836","nonce":4002126418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4253,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.23:6837/1693839903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6835","nonce":1693839903},{"type":"v1","addr":"172.21.15.23:6837","nonce":1693839903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4294,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/357813579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":357813579},{"type":"v1","addr":"172.21.15.112:6836","nonce":357813579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4297,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/394077155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":394077155},{"type":"v1","addr":"172.21.15.112:6837","nonce":394077155}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4300,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/538239518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":538239518},{"type":"v1","addr":"172.21.15.112:6839","nonce":538239518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4306,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.23:6833/1046971212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6832","nonce":1046971212},{"type":"v1","addr":"172.21.15.23:6833","nonce":1046971212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-03-27T10:02:08.539 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.540+0000 7f941b9ff700 1 -- 172.21.15.23:0/603728337 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f93f8066e30 msgr2=0x7f93f80692e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:08.539 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.540+0000 7f941b9ff700 1 --2- 172.21.15.23:0/603728337 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f93f8066e30 0x7f93f80692e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f941414aa60 tx=0x7f9404009210 comp rx=0 tx=0).stop 2023-03-27T10:02:08.545 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.540+0000 7f941b9ff700 1 -- 172.21.15.23:0/603728337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94140c1c10 msgr2=0x7f94141f1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:08.545 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.540+0000 7f941b9ff700 1 --2- 172.21.15.23:0/603728337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94140c1c10 0x7f94141f1510 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9410009c10 tx=0x7f9410004750 comp rx=0 tx=0).stop 2023-03-27T10:02:08.545 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.540+0000 7f941b9ff700 1 -- 172.21.15.23:0/603728337 shutdown_connections 2023-03-27T10:02:08.545 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.540+0000 7f941b9ff700 1 --2- 172.21.15.23:0/603728337 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f941414f840 0x7f9414144270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:08.546 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.540+0000 7f941b9ff700 1 --2- 172.21.15.23:0/603728337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94140c1c10 0x7f94141f1510 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:08.546 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.540+0000 7f941b9ff700 1 --2- 172.21.15.23:0/603728337 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f93f8066e30 0x7f93f80692e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:08.546 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.540+0000 7f941b9ff700 1 --2- 172.21.15.23:0/603728337 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f94140c6430 0x7f94140c43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:08.546 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.540+0000 7f941b9ff700 1 -- 172.21.15.23:0/603728337 >> 172.21.15.23:0/603728337 conn(0x7f941406a1f0 msgr2=0x7f9414070bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:08.546 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.540+0000 7f941b9ff700 1 -- 172.21.15.23:0/603728337 shutdown_connections 2023-03-27T10:02:08.546 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.540+0000 7f941b9ff700 1 -- 172.21.15.23:0/603728337 wait complete. 2023-03-27T10:02:08.546 INFO:teuthology.orchestra.run.smithi023.stderr:dumped fsmap epoch 3 2023-03-27T10:02:08.557 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-27T10:02:08.745 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.746+0000 7f237adff700 1 Processor -- start 2023-03-27T10:02:08.746 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.746+0000 7f237adff700 1 -- start start 2023-03-27T10:02:08.747 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.747+0000 7f237adff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f23740c6990 0x7f23740c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:08.747 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.747+0000 7f237adff700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f237406b440 0x7f237406b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:08.747 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.747+0000 7f237adff700 1 -- --> v1:172.21.15.23:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23740ca730 con 0x7f23740c9520 2023-03-27T10:02:08.747 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.747+0000 7f237adff700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23740c72a0 con 0x7f237406b440 2023-03-27T10:02:08.747 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.747+0000 7f237adff700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23740cf9f0 con 0x7f23740c6990 2023-03-27T10:02:08.748 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.748+0000 7f237939c700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f237406b440 0x7f237406b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:08.748 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.748+0000 7f2373fff700 1 --1- >> v1:172.21.15.23:6789/0 conn(0x7f23740c9520 0x7f23740c8820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.23:6789/0 says I am v1:172.21.15.23:45840/0 (socket says 172.21.15.23:45840) 2023-03-27T10:02:08.748 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.748+0000 7f2373fff700 1 -- 172.21.15.23:0/2719867601 learned_addr learned my addr 172.21.15.23:0/2719867601 (peer_addr_for_me v1:172.21.15.23:0/0) 2023-03-27T10:02:08.748 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.748+0000 7f2378b9b700 1 --2- 172.21.15.23:0/2719867601 >> v2:172.21.15.112:3300/0 conn(0x7f23740c6990 0x7f23740c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:08.749 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.748+0000 7f237939c700 1 -- 172.21.15.23:0/2719867601 >> v2:172.21.15.112:3300/0 conn(0x7f23740c6990 msgr2=0x7f23740c6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:02:08.749 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.748+0000 7f237939c700 1 --2- 172.21.15.23:0/2719867601 >> v2:172.21.15.112:3300/0 conn(0x7f23740c6990 0x7f23740c6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:08.749 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.748+0000 7f237939c700 1 -- 172.21.15.23:0/2719867601 >> v1:172.21.15.23:6789/0 conn(0x7f23740c9520 legacy=0x7f23740c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:08.750 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.748+0000 7f237939c700 1 -- 172.21.15.23:0/2719867601 --> v2:172.21.15.23:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23740c8f30 con 0x7f237406b440 2023-03-27T10:02:08.750 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.748+0000 7f237939c700 1 --2- 172.21.15.23:0/2719867601 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f237406b440 0x7f237406b810 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2374072420 tx=0x7f236c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f9c555782ded0a5 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:08.750 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.748+0000 7f23737fe700 1 -- 172.21.15.23:0/2719867601 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c00e070 con 0x7f237406b440 2023-03-27T10:02:08.750 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.749+0000 7f23737fe700 1 -- 172.21.15.23:0/2719867601 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236c00bb20 con 0x7f237406b440 2023-03-27T10:02:08.750 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.749+0000 7f23737fe700 1 -- 172.21.15.23:0/2719867601 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c012430 con 0x7f237406b440 2023-03-27T10:02:08.750 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.749+0000 7f237adff700 1 -- 172.21.15.23:0/2719867601 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f237406b440 msgr2=0x7f237406b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:08.750 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.749+0000 7f237adff700 1 --2- 172.21.15.23:0/2719867601 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f237406b440 0x7f237406b810 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2374072420 tx=0x7f236c00b330 comp rx=0 tx=0).stop 2023-03-27T10:02:08.750 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.749+0000 7f237adff700 1 -- 172.21.15.23:0/2719867601 shutdown_connections 2023-03-27T10:02:08.751 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.749+0000 7f237adff700 1 --2- 172.21.15.23:0/2719867601 >> v2:172.21.15.112:3300/0 conn(0x7f23740c6990 0x7f23740c6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:08.751 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.749+0000 7f237adff700 1 --2- 172.21.15.23:0/2719867601 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f237406b440 0x7f237406b810 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:08.751 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.749+0000 7f237adff700 1 -- 172.21.15.23:0/2719867601 >> 172.21.15.23:0/2719867601 conn(0x7f2374071920 msgr2=0x7f2374071cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:08.751 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.749+0000 7f237adff700 1 -- 172.21.15.23:0/2719867601 shutdown_connections 2023-03-27T10:02:08.751 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.749+0000 7f237adff700 1 -- 172.21.15.23:0/2719867601 wait complete. 2023-03-27T10:02:08.753 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.751+0000 7f237adff700 1 Processor -- start 2023-03-27T10:02:08.753 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.751+0000 7f237adff700 1 -- start start 2023-03-27T10:02:08.753 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.752+0000 7f237adff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f237406b440 0x7f23741e4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:08.753 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.752+0000 7f237adff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23740c6990 0x7f237414de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:08.753 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.752+0000 7f237adff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f23740c9520 0x7f237414e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:08.754 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.752+0000 7f237adff700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23741e2470 con 0x7f23740c9520 2023-03-27T10:02:08.754 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.752+0000 7f237adff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23741e35f0 con 0x7f23740c6990 2023-03-27T10:02:08.754 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.752+0000 7f237adff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23741e2220 con 0x7f237406b440 2023-03-27T10:02:08.754 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.752+0000 7f237939c700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f23740c9520 0x7f237414e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:08.754 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.752+0000 7f2378b9b700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f237406b440 0x7f23741e4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:08.754 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.752+0000 7f2373fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23740c6990 0x7f237414de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:08.754 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.752+0000 7f237939c700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f23740c9520 0x7f237414e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:48672/0 (socket says 172.21.15.23:48672) 2023-03-27T10:02:08.755 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.752+0000 7f2378b9b700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f237406b440 0x7f23741e4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.23:48992/0 (socket says 172.21.15.23:48992) 2023-03-27T10:02:08.755 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.752+0000 7f237939c700 1 -- 172.21.15.23:0/913504883 learned_addr learned my addr 172.21.15.23:0/913504883 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:08.755 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.753+0000 7f237939c700 1 -- 172.21.15.23:0/913504883 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f237406b440 msgr2=0x7f23741e4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:08.755 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.753+0000 7f237939c700 1 --2- 172.21.15.23:0/913504883 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f237406b440 0x7f23741e4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:08.755 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.753+0000 7f237939c700 1 -- 172.21.15.23:0/913504883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23740c6990 msgr2=0x7f237414de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:08.755 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.753+0000 7f237939c700 1 --2- 172.21.15.23:0/913504883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23740c6990 0x7f237414de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:08.755 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.753+0000 7f237939c700 1 -- 172.21.15.23:0/913504883 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236c009a70 con 0x7f23740c9520 2023-03-27T10:02:08.755 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.753+0000 7f237939c700 1 --2- 172.21.15.23:0/913504883 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f23740c9520 0x7f237414e370 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f236c014040 tx=0x7f236c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:08.756 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.753+0000 7f23717fa700 1 -- 172.21.15.23:0/913504883 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c012430 con 0x7f23740c9520 2023-03-27T10:02:08.756 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.753+0000 7f23717fa700 1 -- 172.21.15.23:0/913504883 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236c020070 con 0x7f23740c9520 2023-03-27T10:02:08.756 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.753+0000 7f237adff700 1 -- 172.21.15.23:0/913504883 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2374158ce0 con 0x7f23740c9520 2023-03-27T10:02:08.756 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.753+0000 7f23717fa700 1 -- 172.21.15.23:0/913504883 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c0098f0 con 0x7f23740c9520 2023-03-27T10:02:08.756 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.753+0000 7f237adff700 1 -- 172.21.15.23:0/913504883 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2374158a00 con 0x7f23740c9520 2023-03-27T10:02:08.757 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.754+0000 7f237adff700 1 -- 172.21.15.23:0/913504883 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2374158fc0 con 0x7f23740c9520 2023-03-27T10:02:08.757 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.755+0000 7f23717fa700 1 -- 172.21.15.23:0/913504883 <== mon.0 v2:172.21.15.23:3300/0 4 ==== mgrmap(e 6) v1 ==== 85113+0+0 (secure 0 0 0) 0x7f236c02e070 con 0x7f23740c9520 2023-03-27T10:02:08.757 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.756+0000 7f23717fa700 1 --2- 172.21.15.23:0/913504883 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f2358066e30 0x7f23580692e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:08.757 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.756+0000 7f23717fa700 1 -- 172.21.15.23:0/913504883 <== mon.0 v2:172.21.15.23:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f236c022070 con 0x7f23740c9520 2023-03-27T10:02:08.758 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.757+0000 7f2378b9b700 1 --2- 172.21.15.23:0/913504883 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f2358066e30 0x7f23580692e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:08.758 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.757+0000 7f2378b9b700 1 --2- 172.21.15.23:0/913504883 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f2358066e30 0x7f23580692e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2368009be0 tx=0x7f2368009210 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:08.760 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.761+0000 7f23717fa700 1 -- 172.21.15.23:0/913504883 <== mon.0 v2:172.21.15.23:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173280 (secure 0 0 0) 0x7f236c053170 con 0x7f23740c9520 2023-03-27T10:02:08.965 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.965+0000 7f237adff700 1 -- 172.21.15.23:0/913504883 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2374069220 con 0x7f23740c9520 2023-03-27T10:02:08.966 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.966+0000 7f23717fa700 1 -- 172.21.15.23:0/913504883 <== mon.0 v2:172.21.15.23:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+14242 (secure 0 0 0) 0x7f236c02c090 con 0x7f23740c9520 2023-03-27T10:02:08.967 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:02:08.967 INFO:teuthology.orchestra.run.smithi023.stdout:{"epoch":16,"fsid":"1d8d6e6e-0d21-4f51-91ee-d9434d4fb750","created":"2023-03-27T10:01:53.805778+0000","modified":"2023-03-27T10:02:07.660358+0000","last_up_change":"2023-03-27T10:02:04.172266+0000","last_in_change":"2023-03-27T10:01:56.475091+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-27T10:02:05.288883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-27T10:02:05.643442+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df086052-536b-4fb1-bcce-7aaf5a13911e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6800","nonce":96642},{"type":"v1","addr":"172.21.15.23:6801","nonce":96642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6802","nonce":96642},{"type":"v1","addr":"172.21.15.23:6803","nonce":96642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6806","nonce":96642},{"type":"v1","addr":"172.21.15.23:6807","nonce":96642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6804","nonce":96642},{"type":"v1","addr":"172.21.15.23:6805","nonce":96642}]},"public_addr":"172.21.15.23:6801/96642","cluster_addr":"172.21.15.23:6803/96642","heartbeat_back_addr":"172.21.15.23:6807/96642","heartbeat_front_addr":"172.21.15.23:6805/96642","state":["exists","up"]},{"osd":1,"uuid":"03138001-a3fd-43fc-9797-a47643d9a3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6808","nonce":96643},{"type":"v1","addr":"172.21.15.23:6809","nonce":96643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6810","nonce":96643},{"type":"v1","addr":"172.21.15.23:6811","nonce":96643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6814","nonce":96643},{"type":"v1","addr":"172.21.15.23:6815","nonce":96643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6812","nonce":96643},{"type":"v1","addr":"172.21.15.23:6813","nonce":96643}]},"public_addr":"172.21.15.23:6809/96643","cluster_addr":"172.21.15.23:6811/96643","heartbeat_back_addr":"172.21.15.23:6815/96643","heartbeat_front_addr":"172.21.15.23:6813/96643","state":["exists","up"]},{"osd":2,"uuid":"964dd926-1160-44ef-aa34-b18965dc05f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6817","nonce":96644},{"type":"v1","addr":"172.21.15.23:6819","nonce":96644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6820","nonce":96644},{"type":"v1","addr":"172.21.15.23:6822","nonce":96644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6827","nonce":96644},{"type":"v1","addr":"172.21.15.23:6829","nonce":96644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6824","nonce":96644},{"type":"v1","addr":"172.21.15.23:6825","nonce":96644}]},"public_addr":"172.21.15.23:6819/96644","cluster_addr":"172.21.15.23:6822/96644","heartbeat_back_addr":"172.21.15.23:6829/96644","heartbeat_front_addr":"172.21.15.23:6825/96644","state":["exists","up"]},{"osd":3,"uuid":"4ae5500f-cb24-44a8-a536-5c399c09e566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6816","nonce":96645},{"type":"v1","addr":"172.21.15.23:6818","nonce":96645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6821","nonce":96645},{"type":"v1","addr":"172.21.15.23:6823","nonce":96645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6830","nonce":96645},{"type":"v1","addr":"172.21.15.23:6831","nonce":96645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6826","nonce":96645},{"type":"v1","addr":"172.21.15.23:6828","nonce":96645}]},"public_addr":"172.21.15.23:6818/96645","cluster_addr":"172.21.15.23:6823/96645","heartbeat_back_addr":"172.21.15.23:6831/96645","heartbeat_front_addr":"172.21.15.23:6828/96645","state":["exists","up"]},{"osd":4,"uuid":"de228ab2-92b4-40bc-8dbd-8f2f4895dc22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":127986},{"type":"v1","addr":"172.21.15.112:6809","nonce":127986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":127986},{"type":"v1","addr":"172.21.15.112:6811","nonce":127986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6815","nonce":127986},{"type":"v1","addr":"172.21.15.112:6818","nonce":127986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":127986},{"type":"v1","addr":"172.21.15.112:6813","nonce":127986}]},"public_addr":"172.21.15.112:6809/127986","cluster_addr":"172.21.15.112:6811/127986","heartbeat_back_addr":"172.21.15.112:6818/127986","heartbeat_front_addr":"172.21.15.112:6813/127986","state":["exists","up"]},{"osd":5,"uuid":"6c839571-b490-4e66-92ac-0d7301c2e677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":127987},{"type":"v1","addr":"172.21.15.112:6801","nonce":127987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":127987},{"type":"v1","addr":"172.21.15.112:6803","nonce":127987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":127987},{"type":"v1","addr":"172.21.15.112:6807","nonce":127987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":127987},{"type":"v1","addr":"172.21.15.112:6805","nonce":127987}]},"public_addr":"172.21.15.112:6801/127987","cluster_addr":"172.21.15.112:6803/127987","heartbeat_back_addr":"172.21.15.112:6807/127987","heartbeat_front_addr":"172.21.15.112:6805/127987","state":["exists","up"]},{"osd":6,"uuid":"871b1a2b-f0ed-4faf-8f9a-a5a0973e11c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":127988},{"type":"v1","addr":"172.21.15.112:6816","nonce":127988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":127988},{"type":"v1","addr":"172.21.15.112:6819","nonce":127988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":127988},{"type":"v1","addr":"172.21.15.112:6823","nonce":127988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":127988},{"type":"v1","addr":"172.21.15.112:6821","nonce":127988}]},"public_addr":"172.21.15.112:6816/127988","cluster_addr":"172.21.15.112:6819/127988","heartbeat_back_addr":"172.21.15.112:6823/127988","heartbeat_front_addr":"172.21.15.112:6821/127988","state":["exists","up"]},{"osd":7,"uuid":"d8228353-34b5-4363-84d8-642512f5bc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":128009},{"type":"v1","addr":"172.21.15.112:6825","nonce":128009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":128009},{"type":"v1","addr":"172.21.15.112:6827","nonce":128009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":128009},{"type":"v1","addr":"172.21.15.112:6831","nonce":128009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":128009},{"type":"v1","addr":"172.21.15.112:6829","nonce":128009}]},"public_addr":"172.21.15.112:6825/128009","cluster_addr":"172.21.15.112:6827/128009","heartbeat_back_addr":"172.21.15.112:6831/128009","heartbeat_front_addr":"172.21.15.112:6829/128009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:02.132160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:01.882517+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:01.927673+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:01.867525+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:01.723580+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:01.730394+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:01.716687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:01.744322+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-27T10:02:08.968 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.969+0000 7f237adff700 1 -- 172.21.15.23:0/913504883 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f2358066e30 msgr2=0x7f23580692e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:08.969 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.969+0000 7f237adff700 1 --2- 172.21.15.23:0/913504883 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f2358066e30 0x7f23580692e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2368009be0 tx=0x7f2368009210 comp rx=0 tx=0).stop 2023-03-27T10:02:08.969 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.969+0000 7f237adff700 1 -- 172.21.15.23:0/913504883 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f23740c9520 msgr2=0x7f237414e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:08.970 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.969+0000 7f237adff700 1 --2- 172.21.15.23:0/913504883 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f23740c9520 0x7f237414e370 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f236c014040 tx=0x7f236c004ad0 comp rx=0 tx=0).stop 2023-03-27T10:02:08.970 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.969+0000 7f237adff700 1 -- 172.21.15.23:0/913504883 shutdown_connections 2023-03-27T10:02:08.970 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.969+0000 7f237adff700 1 --2- 172.21.15.23:0/913504883 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f23740c9520 0x7f237414e370 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:08.971 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.969+0000 7f237adff700 1 --2- 172.21.15.23:0/913504883 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f2358066e30 0x7f23580692e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:08.971 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.969+0000 7f237adff700 1 --2- 172.21.15.23:0/913504883 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f237406b440 0x7f23741e4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:08.971 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.969+0000 7f237adff700 1 --2- 172.21.15.23:0/913504883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23740c6990 0x7f237414de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:08.971 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.969+0000 7f237adff700 1 -- 172.21.15.23:0/913504883 >> 172.21.15.23:0/913504883 conn(0x7f2374071920 msgr2=0x7f23740ce1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:08.972 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.969+0000 7f237adff700 1 -- 172.21.15.23:0/913504883 shutdown_connections 2023-03-27T10:02:08.972 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:08.969+0000 7f237adff700 1 -- 172.21.15.23:0/913504883 wait complete. 2023-03-27T10:02:08.987 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-03-27T10:02:05.288883+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-03-27T10:02:05.643442+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-03-27T10:02:08.988 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-27T10:02:09.180 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.180+0000 7f4b0dfff700 1 Processor -- start 2023-03-27T10:02:09.180 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.181+0000 7f4b0dfff700 1 -- start start 2023-03-27T10:02:09.182 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.182+0000 7f4b0dfff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4b080c6990 0x7f4b080c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:09.182 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.182+0000 7f4b0dfff700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f4b080c9520 0x7f4b080c8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:09.182 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.182+0000 7f4b0dfff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b080ca730 con 0x7f4b0806b440 2023-03-27T10:02:09.182 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.182+0000 7f4b0dfff700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b080c72a0 con 0x7f4b080c9520 2023-03-27T10:02:09.183 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.182+0000 7f4b0dfff700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b080cf9f0 con 0x7f4b080c6990 2023-03-27T10:02:09.183 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.182+0000 7f4afffff700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f4b080c9520 0x7f4b080c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:09.183 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.182+0000 7f4afffff700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f4b080c9520 0x7f4b080c8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:48676/0 (socket says 172.21.15.23:48676) 2023-03-27T10:02:09.183 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.182+0000 7f4afffff700 1 -- 172.21.15.23:0/443591815 learned_addr learned my addr 172.21.15.23:0/443591815 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:09.183 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.182+0000 7f4b077fe700 1 --2- 172.21.15.23:0/443591815 >> v2:172.21.15.112:3300/0 conn(0x7f4b080c6990 0x7f4b080c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:09.183 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.182+0000 7f4afffff700 1 -- 172.21.15.23:0/443591815 >> v2:172.21.15.112:3300/0 conn(0x7f4b080c6990 msgr2=0x7f4b080c6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:02:09.183 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.182+0000 7f4afffff700 1 --2- 172.21.15.23:0/443591815 >> v2:172.21.15.112:3300/0 conn(0x7f4b080c6990 0x7f4b080c6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:09.183 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.182+0000 7f4afffff700 1 -- 172.21.15.23:0/443591815 >> v1:172.21.15.112:6789/0 conn(0x7f4b0806b440 legacy=0x7f4b0806b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:09.184 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.182+0000 7f4afffff700 1 -- 172.21.15.23:0/443591815 --> v2:172.21.15.23:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b08158a00 con 0x7f4b080c9520 2023-03-27T10:02:09.184 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.183+0000 7f4afffff700 1 --2- 172.21.15.23:0/443591815 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f4b080c9520 0x7f4b080c8820 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4b08072420 tx=0x7f4af0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=813bdabb7c3d7cce server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:09.185 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.183+0000 7f4b06ffd700 1 -- 172.21.15.23:0/443591815 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af0011830 con 0x7f4b080c9520 2023-03-27T10:02:09.185 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.183+0000 7f4b06ffd700 1 -- 172.21.15.23:0/443591815 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af0011990 con 0x7f4b080c9520 2023-03-27T10:02:09.185 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.183+0000 7f4b06ffd700 1 -- 172.21.15.23:0/443591815 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af0011d20 con 0x7f4b080c9520 2023-03-27T10:02:09.186 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.183+0000 7f4b0dfff700 1 -- 172.21.15.23:0/443591815 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f4b080c9520 msgr2=0x7f4b080c8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:09.186 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.183+0000 7f4b0dfff700 1 --2- 172.21.15.23:0/443591815 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f4b080c9520 0x7f4b080c8820 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4b08072420 tx=0x7f4af0009bd0 comp rx=0 tx=0).stop 2023-03-27T10:02:09.186 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.183+0000 7f4b0dfff700 1 -- 172.21.15.23:0/443591815 shutdown_connections 2023-03-27T10:02:09.186 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.183+0000 7f4b0dfff700 1 --2- 172.21.15.23:0/443591815 >> v2:172.21.15.112:3300/0 conn(0x7f4b080c6990 0x7f4b080c6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:09.186 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.183+0000 7f4b0dfff700 1 --2- 172.21.15.23:0/443591815 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f4b080c9520 0x7f4b080c8820 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:09.187 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.183+0000 7f4b0dfff700 1 -- 172.21.15.23:0/443591815 >> 172.21.15.23:0/443591815 conn(0x7f4b08071920 msgr2=0x7f4b08071cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:09.187 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.184+0000 7f4b0dfff700 1 -- 172.21.15.23:0/443591815 shutdown_connections 2023-03-27T10:02:09.187 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.184+0000 7f4b0dfff700 1 -- 172.21.15.23:0/443591815 wait complete. 2023-03-27T10:02:09.187 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.185+0000 7f4b0dfff700 1 Processor -- start 2023-03-27T10:02:09.187 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.185+0000 7f4b0dfff700 1 -- start start 2023-03-27T10:02:09.189 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.186+0000 7f4b0dfff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f4b0806b440 0x7f4b0815b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:09.190 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.186+0000 7f4b0dfff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b080c6990 0x7f4b08157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:09.190 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.186+0000 7f4b0dfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b080c9520 0x7f4b081580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:09.190 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.186+0000 7f4b0dfff700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0815b1a0 con 0x7f4b0806b440 2023-03-27T10:02:09.190 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.186+0000 7f4b0dfff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b08153790 con 0x7f4b080c9520 2023-03-27T10:02:09.190 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.186+0000 7f4b0dfff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b081eeb30 con 0x7f4b080c6990 2023-03-27T10:02:09.190 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.186+0000 7f4b077fe700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f4b0806b440 0x7f4b0815b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:09.191 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.186+0000 7f4b077fe700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f4b0806b440 0x7f4b0815b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:48682/0 (socket says 172.21.15.23:48682) 2023-03-27T10:02:09.191 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.186+0000 7f4b077fe700 1 -- 172.21.15.23:0/1771011487 learned_addr learned my addr 172.21.15.23:0/1771011487 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:09.191 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.187+0000 7f4afffff700 1 --2- 172.21.15.23:0/1771011487 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b080c6990 0x7f4b08157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:09.191 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.187+0000 7f4b07fff700 1 --2- 172.21.15.23:0/1771011487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b080c9520 0x7f4b081580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:09.191 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.187+0000 7f4b077fe700 1 -- 172.21.15.23:0/1771011487 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b080c6990 msgr2=0x7f4b08157b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:09.191 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.187+0000 7f4b077fe700 1 --2- 172.21.15.23:0/1771011487 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b080c6990 0x7f4b08157b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:09.192 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.187+0000 7f4b077fe700 1 -- 172.21.15.23:0/1771011487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b080c9520 msgr2=0x7f4b081580d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:09.192 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.187+0000 7f4b077fe700 1 --2- 172.21.15.23:0/1771011487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b080c9520 0x7f4b081580d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:09.192 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.187+0000 7f4b077fe700 1 -- 172.21.15.23:0/1771011487 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af0009550 con 0x7f4b0806b440 2023-03-27T10:02:09.192 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.187+0000 7f4b077fe700 1 --2- 172.21.15.23:0/1771011487 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f4b0806b440 0x7f4b0815b5c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4af800e9b0 tx=0x7f4af800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:09.192 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.187+0000 7f4b04ff9700 1 -- 172.21.15.23:0/1771011487 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af8009710 con 0x7f4b0806b440 2023-03-27T10:02:09.192 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.187+0000 7f4b04ff9700 1 -- 172.21.15.23:0/1771011487 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af800ee50 con 0x7f4b0806b440 2023-03-27T10:02:09.192 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.187+0000 7f4b0dfff700 1 -- 172.21.15.23:0/1771011487 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b08158fc0 con 0x7f4b0806b440 2023-03-27T10:02:09.192 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.187+0000 7f4b04ff9700 1 -- 172.21.15.23:0/1771011487 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af800ca50 con 0x7f4b0806b440 2023-03-27T10:02:09.193 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.187+0000 7f4b0dfff700 1 -- 172.21.15.23:0/1771011487 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b08158ce0 con 0x7f4b0806b440 2023-03-27T10:02:09.193 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.188+0000 7f4b0dfff700 1 -- 172.21.15.23:0/1771011487 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b081592a0 con 0x7f4b0806b440 2023-03-27T10:02:09.193 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.189+0000 7f4b04ff9700 1 -- 172.21.15.23:0/1771011487 <== mon.0 v2:172.21.15.23:3300/0 4 ==== mgrmap(e 6) v1 ==== 85113+0+0 (secure 0 0 0) 0x7f4af8019030 con 0x7f4b0806b440 2023-03-27T10:02:09.193 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.190+0000 7f4b04ff9700 1 --2- 172.21.15.23:0/1771011487 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f4ae8066e30 0x7f4ae80692e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:09.193 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.190+0000 7f4b04ff9700 1 -- 172.21.15.23:0/1771011487 <== mon.0 v2:172.21.15.23:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f4af8083da0 con 0x7f4b0806b440 2023-03-27T10:02:09.194 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.191+0000 7f4afffff700 1 --2- 172.21.15.23:0/1771011487 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f4ae8066e30 0x7f4ae80692e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:09.194 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.191+0000 7f4afffff700 1 --2- 172.21.15.23:0/1771011487 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f4ae8066e30 0x7f4ae80692e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4af0014040 tx=0x7f4af0008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:09.194 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.195+0000 7f4b04ff9700 1 -- 172.21.15.23:0/1771011487 <== mon.0 v2:172.21.15.23:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173280 (secure 0 0 0) 0x7f4af8051330 con 0x7f4b0806b440 2023-03-27T10:02:09.399 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.400+0000 7f4b0dfff700 1 -- 172.21.15.23:0/1771011487 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4b0815b410 con 0x7f4b0806b440 2023-03-27T10:02:09.400 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.400+0000 7f4b04ff9700 1 -- 172.21.15.23:0/1771011487 <== mon.0 v2:172.21.15.23:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4af8054950 con 0x7f4b0806b440 2023-03-27T10:02:09.400 INFO:teuthology.orchestra.run.smithi023.stdout:pg_num: 1 2023-03-27T10:02:09.402 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.402+0000 7f4b0dfff700 1 -- 172.21.15.23:0/1771011487 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f4ae8066e30 msgr2=0x7f4ae80692e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:09.402 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.402+0000 7f4b0dfff700 1 --2- 172.21.15.23:0/1771011487 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f4ae8066e30 0x7f4ae80692e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4af0014040 tx=0x7f4af0008030 comp rx=0 tx=0).stop 2023-03-27T10:02:09.402 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.402+0000 7f4b0dfff700 1 -- 172.21.15.23:0/1771011487 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f4b0806b440 msgr2=0x7f4b0815b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:09.402 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.402+0000 7f4b0dfff700 1 --2- 172.21.15.23:0/1771011487 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f4b0806b440 0x7f4b0815b5c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4af800e9b0 tx=0x7f4af800ecc0 comp rx=0 tx=0).stop 2023-03-27T10:02:09.403 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.402+0000 7f4b0dfff700 1 -- 172.21.15.23:0/1771011487 shutdown_connections 2023-03-27T10:02:09.403 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.402+0000 7f4b0dfff700 1 --2- 172.21.15.23:0/1771011487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b080c9520 0x7f4b081580d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:09.403 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.402+0000 7f4b0dfff700 1 --2- 172.21.15.23:0/1771011487 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f4b0806b440 0x7f4b0815b5c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:09.403 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.402+0000 7f4b0dfff700 1 --2- 172.21.15.23:0/1771011487 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f4ae8066e30 0x7f4ae80692e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:09.403 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.402+0000 7f4b0dfff700 1 --2- 172.21.15.23:0/1771011487 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b080c6990 0x7f4b08157b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:09.404 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.402+0000 7f4b0dfff700 1 -- 172.21.15.23:0/1771011487 >> 172.21.15.23:0/1771011487 conn(0x7f4b08071920 msgr2=0x7f4b08071cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:09.404 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.403+0000 7f4b0dfff700 1 -- 172.21.15.23:0/1771011487 shutdown_connections 2023-03-27T10:02:09.404 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.403+0000 7f4b0dfff700 1 -- 172.21.15.23:0/1771011487 wait complete. 2023-03-27T10:02:09.420 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-27T10:02:09.610 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.610+0000 7f76987ff700 1 Processor -- start 2023-03-27T10:02:09.610 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.610+0000 7f76987ff700 1 -- start start 2023-03-27T10:02:09.612 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.611+0000 7f76987ff700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f76900c6990 0x7f76900c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:09.612 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.611+0000 7f76987ff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f76900c9520 0x7f76900c8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:09.612 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.611+0000 7f76987ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f769006b440 0x7f769006b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:09.613 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.612+0000 7f76987ff700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76900ca730 con 0x7f76900c6990 2023-03-27T10:02:09.614 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.612+0000 7f76987ff700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76900c72a0 con 0x7f76900c9520 2023-03-27T10:02:09.614 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.612+0000 7f76987ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76900cf9f0 con 0x7f769006b440 2023-03-27T10:02:09.614 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.612+0000 7f769659b700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f76900c6990 0x7f76900c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:09.614 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.612+0000 7f769659b700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f76900c6990 0x7f76900c6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:48694/0 (socket says 172.21.15.23:48694) 2023-03-27T10:02:09.615 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.612+0000 7f769659b700 1 -- 172.21.15.23:0/5459531 learned_addr learned my addr 172.21.15.23:0/5459531 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:09.615 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.612+0000 7f7695d9a700 1 --2- 172.21.15.23:0/5459531 >> v2:172.21.15.112:3300/0 conn(0x7f76900c9520 0x7f76900c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:09.615 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.612+0000 7f7696d9c700 1 --2- 172.21.15.23:0/5459531 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f769006b440 0x7f769006b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:09.615 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.612+0000 7f769659b700 1 -- 172.21.15.23:0/5459531 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f769006b440 msgr2=0x7f769006b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:02:09.616 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.612+0000 7f769659b700 1 --2- 172.21.15.23:0/5459531 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f769006b440 0x7f769006b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:09.616 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.612+0000 7f769659b700 1 -- 172.21.15.23:0/5459531 >> v2:172.21.15.112:3300/0 conn(0x7f76900c9520 msgr2=0x7f76900c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:02:09.616 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.612+0000 7f769659b700 1 --2- 172.21.15.23:0/5459531 >> v2:172.21.15.112:3300/0 conn(0x7f76900c9520 0x7f76900c8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:09.616 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.612+0000 7f769659b700 1 -- 172.21.15.23:0/5459531 --> v2:172.21.15.23:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7690158a00 con 0x7f76900c6990 2023-03-27T10:02:09.616 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.612+0000 7f7696d9c700 1 --2- 172.21.15.23:0/5459531 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f769006b440 0x7f769006b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-27T10:02:09.617 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.612+0000 7f769659b700 1 --2- 172.21.15.23:0/5459531 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f76900c6990 0x7f76900c6d60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f76900cca50 tx=0x7f768c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24477ecd928e786a server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:09.617 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.613+0000 7f7695599700 1 -- 172.21.15.23:0/5459531 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c011830 con 0x7f76900c6990 2023-03-27T10:02:09.617 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.613+0000 7f7695599700 1 -- 172.21.15.23:0/5459531 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f768c011990 con 0x7f76900c6990 2023-03-27T10:02:09.617 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.613+0000 7f7695599700 1 -- 172.21.15.23:0/5459531 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c011d20 con 0x7f76900c6990 2023-03-27T10:02:09.621 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.613+0000 7f76987ff700 1 -- 172.21.15.23:0/5459531 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f76900c6990 msgr2=0x7f76900c6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:09.622 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.613+0000 7f76987ff700 1 --2- 172.21.15.23:0/5459531 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f76900c6990 0x7f76900c6d60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f76900cca50 tx=0x7f768c009bd0 comp rx=0 tx=0).stop 2023-03-27T10:02:09.622 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.613+0000 7f76987ff700 1 -- 172.21.15.23:0/5459531 shutdown_connections 2023-03-27T10:02:09.622 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.613+0000 7f76987ff700 1 --2- 172.21.15.23:0/5459531 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f769006b440 0x7f769006b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:09.622 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.613+0000 7f76987ff700 1 --2- 172.21.15.23:0/5459531 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f76900c6990 0x7f76900c6d60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:09.623 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.613+0000 7f76987ff700 1 --2- 172.21.15.23:0/5459531 >> v2:172.21.15.112:3300/0 conn(0x7f76900c9520 0x7f76900c8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:09.623 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.613+0000 7f76987ff700 1 -- 172.21.15.23:0/5459531 >> 172.21.15.23:0/5459531 conn(0x7f7690071920 msgr2=0x7f7690071cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:09.623 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.613+0000 7f76987ff700 1 -- 172.21.15.23:0/5459531 shutdown_connections 2023-03-27T10:02:09.623 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.613+0000 7f76987ff700 1 -- 172.21.15.23:0/5459531 wait complete. 2023-03-27T10:02:09.623 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.615+0000 7f76987ff700 1 Processor -- start 2023-03-27T10:02:09.624 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.615+0000 7f76987ff700 1 -- start start 2023-03-27T10:02:09.624 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.616+0000 7f76987ff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f769006b440 0x7f76901eeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:09.624 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.616+0000 7f76987ff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76900c6990 0x7f769015c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:09.624 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.616+0000 7f76987ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f76900c9520 0x7f769014a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:09.624 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.616+0000 7f76987ff700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76900cd6a0 con 0x7f769006b440 2023-03-27T10:02:09.624 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.616+0000 7f76987ff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76900cac40 con 0x7f76900c6990 2023-03-27T10:02:09.625 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.616+0000 7f76987ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76901f2520 con 0x7f76900c9520 2023-03-27T10:02:09.625 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.616+0000 7f769659b700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f769006b440 0x7f76901eeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:09.625 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.616+0000 7f769659b700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f769006b440 0x7f76901eeb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:48698/0 (socket says 172.21.15.23:48698) 2023-03-27T10:02:09.625 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.616+0000 7f7695d9a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76900c6990 0x7f769015c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:09.625 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.616+0000 7f769659b700 1 -- 172.21.15.23:0/973950327 learned_addr learned my addr 172.21.15.23:0/973950327 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:09.625 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.616+0000 7f7696d9c700 1 --2- 172.21.15.23:0/973950327 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f76900c9520 0x7f769014a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:09.626 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.617+0000 7f7695d9a700 1 -- 172.21.15.23:0/973950327 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f76900c9520 msgr2=0x7f769014a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:09.626 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.617+0000 7f7695d9a700 1 --2- 172.21.15.23:0/973950327 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f76900c9520 0x7f769014a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:09.626 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.617+0000 7f7695d9a700 1 -- 172.21.15.23:0/973950327 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f769006b440 msgr2=0x7f76901eeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:09.626 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.617+0000 7f7695d9a700 1 --2- 172.21.15.23:0/973950327 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f769006b440 0x7f76901eeb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:09.626 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.617+0000 7f7695d9a700 1 -- 172.21.15.23:0/973950327 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f768c009550 con 0x7f76900c6990 2023-03-27T10:02:09.626 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.617+0000 7f7695d9a700 1 --2- 172.21.15.23:0/973950327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76900c6990 0x7f769015c010 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f768000e870 tx=0x7f768000eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:09.626 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.617+0000 7f7686ffd700 1 -- 172.21.15.23:0/973950327 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7680013680 con 0x7f76900c6990 2023-03-27T10:02:09.627 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.617+0000 7f7686ffd700 1 -- 172.21.15.23:0/973950327 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76800137e0 con 0x7f76900c6990 2023-03-27T10:02:09.627 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.617+0000 7f76987ff700 1 -- 172.21.15.23:0/973950327 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7690158fc0 con 0x7f76900c6990 2023-03-27T10:02:09.627 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.617+0000 7f7686ffd700 1 -- 172.21.15.23:0/973950327 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7680013b90 con 0x7f76900c6990 2023-03-27T10:02:09.627 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.617+0000 7f76987ff700 1 -- 172.21.15.23:0/973950327 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7690158ce0 con 0x7f76900c6990 2023-03-27T10:02:09.627 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.618+0000 7f76987ff700 1 -- 172.21.15.23:0/973950327 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76901592a0 con 0x7f76900c6990 2023-03-27T10:02:09.627 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.620+0000 7f7686ffd700 1 -- 172.21.15.23:0/973950327 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 85113+0+0 (secure 0 0 0) 0x7f7680019030 con 0x7f76900c6990 2023-03-27T10:02:09.627 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.620+0000 7f7686ffd700 1 --2- 172.21.15.23:0/973950327 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f7674066e30 0x7f76740692e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:09.628 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.620+0000 7f7686ffd700 1 -- 172.21.15.23:0/973950327 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7680083c90 con 0x7f76900c6990 2023-03-27T10:02:09.628 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.621+0000 7f769659b700 1 --2- 172.21.15.23:0/973950327 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f7674066e30 0x7f76740692e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:09.628 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.621+0000 7f769659b700 1 --2- 172.21.15.23:0/973950327 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f7674066e30 0x7f76740692e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f768c009fd0 tx=0x7f768c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:09.630 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.624+0000 7f7686ffd700 1 -- 172.21.15.23:0/973950327 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173280 (secure 0 0 0) 0x7f76800512a0 con 0x7f76900c6990 2023-03-27T10:02:09.826 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.827+0000 7f76987ff700 1 -- 172.21.15.23:0/973950327 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7690069220 con 0x7f76900c6990 2023-03-27T10:02:09.827 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.827+0000 7f7686ffd700 1 -- 172.21.15.23:0/973950327 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v16) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7680017020 con 0x7f76900c6990 2023-03-27T10:02:09.827 INFO:teuthology.orchestra.run.smithi023.stdout:pg_num: 8 2023-03-27T10:02:09.829 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.830+0000 7f76987ff700 1 -- 172.21.15.23:0/973950327 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f7674066e30 msgr2=0x7f76740692e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:09.830 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.830+0000 7f76987ff700 1 --2- 172.21.15.23:0/973950327 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f7674066e30 0x7f76740692e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f768c009fd0 tx=0x7f768c008030 comp rx=0 tx=0).stop 2023-03-27T10:02:09.830 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.830+0000 7f76987ff700 1 -- 172.21.15.23:0/973950327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76900c6990 msgr2=0x7f769015c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:09.830 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.830+0000 7f76987ff700 1 --2- 172.21.15.23:0/973950327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76900c6990 0x7f769015c010 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f768000e870 tx=0x7f768000eb80 comp rx=0 tx=0).stop 2023-03-27T10:02:09.830 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.830+0000 7f76987ff700 1 -- 172.21.15.23:0/973950327 shutdown_connections 2023-03-27T10:02:09.830 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.830+0000 7f76987ff700 1 --2- 172.21.15.23:0/973950327 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f7674066e30 0x7f76740692e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:09.830 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.830+0000 7f76987ff700 1 --2- 172.21.15.23:0/973950327 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f76900c9520 0x7f769014a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:09.831 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.830+0000 7f76987ff700 1 --2- 172.21.15.23:0/973950327 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f769006b440 0x7f76901eeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:09.831 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.830+0000 7f76987ff700 1 --2- 172.21.15.23:0/973950327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76900c6990 0x7f769015c010 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:09.831 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.830+0000 7f76987ff700 1 -- 172.21.15.23:0/973950327 >> 172.21.15.23:0/973950327 conn(0x7f7690071920 msgr2=0x7f76900cb3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:09.833 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.830+0000 7f76987ff700 1 -- 172.21.15.23:0/973950327 shutdown_connections 2023-03-27T10:02:09.833 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:09.830+0000 7f76987ff700 1 -- 172.21.15.23:0/973950327 wait complete. 2023-03-27T10:02:09.846 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-03-27T10:02:10.034 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.034+0000 7f749c7ff700 1 Processor -- start 2023-03-27T10:02:10.034 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.034+0000 7f749c7ff700 1 -- start start 2023-03-27T10:02:10.037 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.035+0000 7f749c7ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74940c9520 0x7f74940c8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:10.038 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.035+0000 7f749c7ff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f749406b440 0x7f749406b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:10.038 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.035+0000 7f749c7ff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74940ca730 con 0x7f74940c6990 2023-03-27T10:02:10.038 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.035+0000 7f749c7ff700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f749406bdc0 con 0x7f749406b440 2023-03-27T10:02:10.039 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.035+0000 7f749c7ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74940cf9f0 con 0x7f74940c9520 2023-03-27T10:02:10.039 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.036+0000 7f7499d9a700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74940c9520 0x7f74940c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:10.039 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.036+0000 7f749ad9c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f749406b440 0x7f749406b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:10.040 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.036+0000 7f749a59b700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f74940c6990 0x7f74940c6d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.23:57246/0 (socket says 172.21.15.23:57246) 2023-03-27T10:02:10.040 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.036+0000 7f749ad9c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f749406b440 0x7f749406b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.23:33838/0 (socket says 172.21.15.23:33838) 2023-03-27T10:02:10.040 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.036+0000 7f749ad9c700 1 -- 172.21.15.23:0/3364095288 learned_addr learned my addr 172.21.15.23:0/3364095288 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:10.040 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.036+0000 7f7499d9a700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74940c9520 0x7f74940c8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.23:49038/0 (socket says 172.21.15.23:49038) 2023-03-27T10:02:10.041 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.036+0000 7f749ad9c700 1 -- 172.21.15.23:0/3364095288 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74940c9520 msgr2=0x7f74940c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:02:10.041 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.036+0000 7f749ad9c700 1 --2- 172.21.15.23:0/3364095288 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74940c9520 0x7f74940c8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:10.041 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.036+0000 7f749ad9c700 1 -- 172.21.15.23:0/3364095288 >> v1:172.21.15.112:6789/0 conn(0x7f74940c6990 legacy=0x7f74940c6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:10.041 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.036+0000 7f749ad9c700 1 -- 172.21.15.23:0/3364095288 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7494158a00 con 0x7f749406b440 2023-03-27T10:02:10.042 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.036+0000 7f7499599700 1 -- 172.21.15.23:0/3364095288 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3233406955 0 0) 0x7f74940ca730 con 0x7f74940c6990 2023-03-27T10:02:10.042 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.036+0000 7f7499d9a700 1 --2- 172.21.15.23:0/3364095288 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74940c9520 0x7f74940c8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-27T10:02:10.042 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.036+0000 7f749ad9c700 1 --2- 172.21.15.23:0/3364095288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f749406b440 0x7f749406b880 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f748c009e50 tx=0x7f748c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a5d31144f10174f server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:10.043 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.036+0000 7f7499599700 1 -- 172.21.15.23:0/3364095288 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c00e070 con 0x7f749406b440 2023-03-27T10:02:10.043 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.037+0000 7f7499599700 1 -- 172.21.15.23:0/3364095288 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748c00b910 con 0x7f749406b440 2023-03-27T10:02:10.043 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.037+0000 7f7499599700 1 -- 172.21.15.23:0/3364095288 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c00bcc0 con 0x7f749406b440 2023-03-27T10:02:10.043 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.037+0000 7f749c7ff700 1 -- 172.21.15.23:0/3364095288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f749406b440 msgr2=0x7f749406b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:10.043 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.037+0000 7f749c7ff700 1 --2- 172.21.15.23:0/3364095288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f749406b440 0x7f749406b880 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f748c009e50 tx=0x7f748c00b330 comp rx=0 tx=0).stop 2023-03-27T10:02:10.043 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.037+0000 7f749c7ff700 1 -- 172.21.15.23:0/3364095288 shutdown_connections 2023-03-27T10:02:10.044 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.037+0000 7f749c7ff700 1 --2- 172.21.15.23:0/3364095288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f749406b440 0x7f749406b880 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:10.044 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.037+0000 7f749c7ff700 1 --2- 172.21.15.23:0/3364095288 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74940c9520 0x7f74940c8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:10.044 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.037+0000 7f749c7ff700 1 -- 172.21.15.23:0/3364095288 >> 172.21.15.23:0/3364095288 conn(0x7f7494071920 msgr2=0x7f7494071cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:10.044 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.037+0000 7f749c7ff700 1 -- 172.21.15.23:0/3364095288 shutdown_connections 2023-03-27T10:02:10.044 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.037+0000 7f749c7ff700 1 -- 172.21.15.23:0/3364095288 wait complete. 2023-03-27T10:02:10.047 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.039+0000 7f749c7ff700 1 Processor -- start 2023-03-27T10:02:10.047 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.039+0000 7f749c7ff700 1 -- start start 2023-03-27T10:02:10.048 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.040+0000 7f749c7ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f749406b440 0x7f7494156c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:10.048 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.040+0000 7f749c7ff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f74940c6990 0x7f7494152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:10.048 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.040+0000 7f749c7ff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74940c9520 0x7f749414f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:10.048 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.040+0000 7f749c7ff700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74941e1530 con 0x7f74940c6990 2023-03-27T10:02:10.048 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.040+0000 7f749c7ff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74941dfc30 con 0x7f74940c9520 2023-03-27T10:02:10.048 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.040+0000 7f749c7ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74941df370 con 0x7f749406b440 2023-03-27T10:02:10.049 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.040+0000 7f7499d9a700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f74940c6990 0x7f7494152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:10.049 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.040+0000 7f749ad9c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74940c9520 0x7f749414f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:10.049 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.040+0000 7f749a59b700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f749406b440 0x7f7494156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:10.049 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.040+0000 7f7499d9a700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f74940c6990 0x7f7494152e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:48712/0 (socket says 172.21.15.23:48712) 2023-03-27T10:02:10.049 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.040+0000 7f7499d9a700 1 -- 172.21.15.23:0/1540212067 learned_addr learned my addr 172.21.15.23:0/1540212067 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:10.050 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.040+0000 7f7499d9a700 1 -- 172.21.15.23:0/1540212067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f749406b440 msgr2=0x7f7494156c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:10.050 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.040+0000 7f7499d9a700 1 --2- 172.21.15.23:0/1540212067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f749406b440 0x7f7494156c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:10.050 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.041+0000 7f7499d9a700 1 -- 172.21.15.23:0/1540212067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74940c9520 msgr2=0x7f749414f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:10.050 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.041+0000 7f7499d9a700 1 --2- 172.21.15.23:0/1540212067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74940c9520 0x7f749414f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:10.050 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.041+0000 7f7499d9a700 1 -- 172.21.15.23:0/1540212067 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74840094a0 con 0x7f74940c6990 2023-03-27T10:02:10.050 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.041+0000 7f749ad9c700 1 --2- 172.21.15.23:0/1540212067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74940c9520 0x7f749414f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-27T10:02:10.051 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.041+0000 7f7499d9a700 1 --2- 172.21.15.23:0/1540212067 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f74940c6990 0x7f7494152e40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f748400ec10 tx=0x7f748400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:10.051 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.041+0000 7f748affd700 1 -- 172.21.15.23:0/1540212067 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7484009760 con 0x7f74940c6990 2023-03-27T10:02:10.051 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.041+0000 7f748affd700 1 -- 172.21.15.23:0/1540212067 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748400c850 con 0x7f74940c6990 2023-03-27T10:02:10.051 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.041+0000 7f748affd700 1 -- 172.21.15.23:0/1540212067 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748400cbe0 con 0x7f74940c6990 2023-03-27T10:02:10.051 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.041+0000 7f749c7ff700 1 -- 172.21.15.23:0/1540212067 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f748c009b30 con 0x7f74940c6990 2023-03-27T10:02:10.051 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.041+0000 7f749c7ff700 1 -- 172.21.15.23:0/1540212067 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7494158fc0 con 0x7f74940c6990 2023-03-27T10:02:10.051 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.042+0000 7f749c7ff700 1 -- 172.21.15.23:0/1540212067 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7494158ce0 con 0x7f74940c6990 2023-03-27T10:02:10.052 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.043+0000 7f748affd700 1 -- 172.21.15.23:0/1540212067 <== mon.0 v2:172.21.15.23:3300/0 4 ==== mgrmap(e 6) v1 ==== 85113+0+0 (secure 0 0 0) 0x7f7484019030 con 0x7f74940c6990 2023-03-27T10:02:10.052 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.044+0000 7f748affd700 1 --2- 172.21.15.23:0/1540212067 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f7478066e60 0x7f7478069310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:10.052 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.044+0000 7f748affd700 1 -- 172.21.15.23:0/1540212067 <== mon.0 v2:172.21.15.23:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7484083db0 con 0x7f74940c6990 2023-03-27T10:02:10.052 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.044+0000 7f749a59b700 1 --2- 172.21.15.23:0/1540212067 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f7478066e60 0x7f7478069310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:10.052 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.045+0000 7f749a59b700 1 --2- 172.21.15.23:0/1540212067 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f7478066e60 0x7f7478069310 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7490007fd0 tx=0x7f7490007de0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:10.052 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.048+0000 7f748affd700 1 -- 172.21.15.23:0/1540212067 <== mon.0 v2:172.21.15.23:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173280 (secure 0 0 0) 0x7f7484088070 con 0x7f74940c6990 2023-03-27T10:02:10.249 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.249+0000 7f749c7ff700 1 -- 172.21.15.23:0/1540212067 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f74941592a0 con 0x7f74940c6990 2023-03-27T10:02:10.249 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.249+0000 7f748affd700 1 -- 172.21.15.23:0/1540212067 <== mon.0 v2:172.21.15.23:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+126 (secure 0 0 0) 0x7f7484054d70 con 0x7f74940c6990 2023-03-27T10:02:10.250 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:02:10.250 INFO:teuthology.orchestra.run.smithi023.stdout:[ 2023-03-27T10:02:10.250 INFO:teuthology.orchestra.run.smithi023.stdout: { 2023-03-27T10:02:10.250 INFO:teuthology.orchestra.run.smithi023.stdout: "poolnum": 1, 2023-03-27T10:02:10.251 INFO:teuthology.orchestra.run.smithi023.stdout: "poolname": ".mgr" 2023-03-27T10:02:10.251 INFO:teuthology.orchestra.run.smithi023.stdout: }, 2023-03-27T10:02:10.251 INFO:teuthology.orchestra.run.smithi023.stdout: { 2023-03-27T10:02:10.251 INFO:teuthology.orchestra.run.smithi023.stdout: "poolnum": 2, 2023-03-27T10:02:10.251 INFO:teuthology.orchestra.run.smithi023.stdout: "poolname": "rbd" 2023-03-27T10:02:10.252 INFO:teuthology.orchestra.run.smithi023.stdout: } 2023-03-27T10:02:10.252 INFO:teuthology.orchestra.run.smithi023.stdout:] 2023-03-27T10:02:10.253 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.251+0000 7f749c7ff700 1 -- 172.21.15.23:0/1540212067 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f7478066e60 msgr2=0x7f7478069310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:10.254 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.251+0000 7f749c7ff700 1 --2- 172.21.15.23:0/1540212067 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f7478066e60 0x7f7478069310 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7490007fd0 tx=0x7f7490007de0 comp rx=0 tx=0).stop 2023-03-27T10:02:10.254 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.251+0000 7f749c7ff700 1 -- 172.21.15.23:0/1540212067 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f74940c6990 msgr2=0x7f7494152e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:10.254 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.251+0000 7f749c7ff700 1 --2- 172.21.15.23:0/1540212067 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f74940c6990 0x7f7494152e40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f748400ec10 tx=0x7f748400ef20 comp rx=0 tx=0).stop 2023-03-27T10:02:10.254 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.252+0000 7f749c7ff700 1 -- 172.21.15.23:0/1540212067 shutdown_connections 2023-03-27T10:02:10.255 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.252+0000 7f749c7ff700 1 --2- 172.21.15.23:0/1540212067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74940c9520 0x7f749414f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:10.255 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.252+0000 7f749c7ff700 1 --2- 172.21.15.23:0/1540212067 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f7478066e60 0x7f7478069310 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:10.256 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.252+0000 7f749c7ff700 1 --2- 172.21.15.23:0/1540212067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f749406b440 0x7f7494156c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:10.256 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.252+0000 7f749c7ff700 1 --2- 172.21.15.23:0/1540212067 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f74940c6990 0x7f7494152e40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:10.256 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.252+0000 7f749c7ff700 1 -- 172.21.15.23:0/1540212067 >> 172.21.15.23:0/1540212067 conn(0x7f7494071920 msgr2=0x7f74940c6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:10.257 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.252+0000 7f749c7ff700 1 -- 172.21.15.23:0/1540212067 shutdown_connections 2023-03-27T10:02:10.257 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.252+0000 7f749c7ff700 1 -- 172.21.15.23:0/1540212067 wait complete. 2023-03-27T10:02:10.269 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-03-27T10:02:10.270 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-03-27T10:02:10.462 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.462+0000 7ff39b7ff700 1 Processor -- start 2023-03-27T10:02:10.462 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.462+0000 7ff39b7ff700 1 -- start start 2023-03-27T10:02:10.463 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.463+0000 7ff39b7ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff3940c8a60 0x7ff394158760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:10.463 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.464+0000 7ff39b7ff700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7ff39415a5f0 0x7ff39415aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:10.463 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.464+0000 7ff39b7ff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff39415afb0 con 0x7ff3940c94b0 2023-03-27T10:02:10.463 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.464+0000 7ff39b7ff700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff39415a1d0 con 0x7ff39415a5f0 2023-03-27T10:02:10.463 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.464+0000 7ff39b7ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff394158ca0 con 0x7ff3940c8a60 2023-03-27T10:02:10.465 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.464+0000 7ff399d9c700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7ff39415a5f0 0x7ff39415aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:10.465 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.464+0000 7ff399d9c700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7ff39415a5f0 0x7ff39415aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:48718/0 (socket says 172.21.15.23:48718) 2023-03-27T10:02:10.465 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.464+0000 7ff399d9c700 1 -- 172.21.15.23:0/4220420299 learned_addr learned my addr 172.21.15.23:0/4220420299 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:10.465 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.464+0000 7ff398d9a700 1 --1- 172.21.15.23:0/4220420299 >> v1:172.21.15.112:6789/0 conn(0x7ff3940c94b0 0x7ff39406ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.23:57254/0 (socket says 172.21.15.23:57254) 2023-03-27T10:02:10.465 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.464+0000 7ff39959b700 1 --2- 172.21.15.23:0/4220420299 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff3940c8a60 0x7ff394158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:10.466 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.464+0000 7ff399d9c700 1 -- 172.21.15.23:0/4220420299 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff3940c8a60 msgr2=0x7ff394158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:02:10.466 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.464+0000 7ff399d9c700 1 --2- 172.21.15.23:0/4220420299 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff3940c8a60 0x7ff394158760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:10.466 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.464+0000 7ff399d9c700 1 -- 172.21.15.23:0/4220420299 >> v1:172.21.15.112:6789/0 conn(0x7ff3940c94b0 legacy=0x7ff39406ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:10.466 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.464+0000 7ff38bfff700 1 -- 172.21.15.23:0/4220420299 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1427555325 0 0) 0x7ff39415afb0 con 0x7ff3940c94b0 2023-03-27T10:02:10.466 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.464+0000 7ff399d9c700 1 -- 172.21.15.23:0/4220420299 --> v2:172.21.15.23:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff394158f30 con 0x7ff39415a5f0 2023-03-27T10:02:10.466 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.465+0000 7ff399d9c700 1 --2- 172.21.15.23:0/4220420299 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7ff39415a5f0 0x7ff39415aa70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff38c009e50 tx=0x7ff38c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c7f77d7b72df463 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:10.466 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.465+0000 7ff38bfff700 1 -- 172.21.15.23:0/4220420299 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38c00e070 con 0x7ff39415a5f0 2023-03-27T10:02:10.467 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.465+0000 7ff38bfff700 1 -- 172.21.15.23:0/4220420299 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff38c00b910 con 0x7ff39415a5f0 2023-03-27T10:02:10.467 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.465+0000 7ff38bfff700 1 -- 172.21.15.23:0/4220420299 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38c00bcc0 con 0x7ff39415a5f0 2023-03-27T10:02:10.467 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.465+0000 7ff39b7ff700 1 -- 172.21.15.23:0/4220420299 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7ff39415a5f0 msgr2=0x7ff39415aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:10.467 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.465+0000 7ff39b7ff700 1 --2- 172.21.15.23:0/4220420299 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7ff39415a5f0 0x7ff39415aa70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff38c009e50 tx=0x7ff38c00b330 comp rx=0 tx=0).stop 2023-03-27T10:02:10.467 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.465+0000 7ff39b7ff700 1 -- 172.21.15.23:0/4220420299 shutdown_connections 2023-03-27T10:02:10.467 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.465+0000 7ff39b7ff700 1 --2- 172.21.15.23:0/4220420299 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7ff39415a5f0 0x7ff39415aa70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:10.467 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.465+0000 7ff39b7ff700 1 --2- 172.21.15.23:0/4220420299 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff3940c8a60 0x7ff394158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:10.467 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.465+0000 7ff39b7ff700 1 -- 172.21.15.23:0/4220420299 >> 172.21.15.23:0/4220420299 conn(0x7ff39406be90 msgr2=0x7ff39406c290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:10.468 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.466+0000 7ff39b7ff700 1 -- 172.21.15.23:0/4220420299 shutdown_connections 2023-03-27T10:02:10.468 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.466+0000 7ff39b7ff700 1 -- 172.21.15.23:0/4220420299 wait complete. 2023-03-27T10:02:10.470 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.467+0000 7ff39b7ff700 1 Processor -- start 2023-03-27T10:02:10.470 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.467+0000 7ff39b7ff700 1 -- start start 2023-03-27T10:02:10.470 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.468+0000 7ff39b7ff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7ff3940c8a60 0x7ff3940cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:10.470 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.468+0000 7ff39b7ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff3940c94b0 0x7ff3940cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:10.470 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.468+0000 7ff39b7ff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff39415a5f0 0x7ff3940b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:10.470 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.468+0000 7ff39b7ff700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff394148230 con 0x7ff3940c8a60 2023-03-27T10:02:10.470 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.468+0000 7ff39b7ff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3940b2740 con 0x7ff39415a5f0 2023-03-27T10:02:10.471 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.468+0000 7ff39b7ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff394148fb0 con 0x7ff3940c94b0 2023-03-27T10:02:10.471 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.468+0000 7ff39959b700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7ff3940c8a60 0x7ff3940cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:10.471 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.468+0000 7ff398d9a700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff3940c94b0 0x7ff3940cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:10.471 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.468+0000 7ff39959b700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7ff3940c8a60 0x7ff3940cf0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:48720/0 (socket says 172.21.15.23:48720) 2023-03-27T10:02:10.471 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.468+0000 7ff39959b700 1 -- 172.21.15.23:0/3200338266 learned_addr learned my addr 172.21.15.23:0/3200338266 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:10.471 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.468+0000 7ff399d9c700 1 --2- 172.21.15.23:0/3200338266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff39415a5f0 0x7ff3940b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:10.471 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.469+0000 7ff39959b700 1 -- 172.21.15.23:0/3200338266 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff3940c94b0 msgr2=0x7ff3940cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:10.472 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.469+0000 7ff39959b700 1 --2- 172.21.15.23:0/3200338266 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff3940c94b0 0x7ff3940cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:10.472 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.469+0000 7ff39959b700 1 -- 172.21.15.23:0/3200338266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff39415a5f0 msgr2=0x7ff3940b2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:10.472 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.469+0000 7ff39959b700 1 --2- 172.21.15.23:0/3200338266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff39415a5f0 0x7ff3940b2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:10.472 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.469+0000 7ff39959b700 1 -- 172.21.15.23:0/3200338266 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff38c009b30 con 0x7ff3940c8a60 2023-03-27T10:02:10.472 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.469+0000 7ff39959b700 1 --2- 172.21.15.23:0/3200338266 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7ff3940c8a60 0x7ff3940cf0b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff39000e8a0 tx=0x7ff39000ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:10.472 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.469+0000 7ff389ffb700 1 -- 172.21.15.23:0/3200338266 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff390009590 con 0x7ff3940c8a60 2023-03-27T10:02:10.472 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.469+0000 7ff389ffb700 1 -- 172.21.15.23:0/3200338266 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff390013680 con 0x7ff3940c8a60 2023-03-27T10:02:10.472 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.469+0000 7ff389ffb700 1 -- 172.21.15.23:0/3200338266 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3900139f0 con 0x7ff3940c8a60 2023-03-27T10:02:10.473 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.469+0000 7ff39b7ff700 1 -- 172.21.15.23:0/3200338266 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3941594f0 con 0x7ff3940c8a60 2023-03-27T10:02:10.473 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.469+0000 7ff39b7ff700 1 -- 172.21.15.23:0/3200338266 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff394159210 con 0x7ff3940c8a60 2023-03-27T10:02:10.473 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.470+0000 7ff39b7ff700 1 -- 172.21.15.23:0/3200338266 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff394159df0 con 0x7ff3940c8a60 2023-03-27T10:02:10.474 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.471+0000 7ff389ffb700 1 -- 172.21.15.23:0/3200338266 <== mon.0 v2:172.21.15.23:3300/0 4 ==== mgrmap(e 6) v1 ==== 85113+0+0 (secure 0 0 0) 0x7ff390019030 con 0x7ff3940c8a60 2023-03-27T10:02:10.474 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.472+0000 7ff389ffb700 1 --2- 172.21.15.23:0/3200338266 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7ff378066e30 0x7ff3780692e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:10.474 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.472+0000 7ff389ffb700 1 -- 172.21.15.23:0/3200338266 <== mon.0 v2:172.21.15.23:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff390083be0 con 0x7ff3940c8a60 2023-03-27T10:02:10.474 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.473+0000 7ff398d9a700 1 --2- 172.21.15.23:0/3200338266 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7ff378066e30 0x7ff3780692e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:10.474 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.473+0000 7ff398d9a700 1 --2- 172.21.15.23:0/3200338266 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7ff378066e30 0x7ff3780692e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff394157180 tx=0x7ff384007de0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:10.475 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.476+0000 7ff389ffb700 1 -- 172.21.15.23:0/3200338266 <== mon.0 v2:172.21.15.23:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173280 (secure 0 0 0) 0x7ff390051170 con 0x7ff3940c8a60 2023-03-27T10:02:10.679 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.680+0000 7ff39b7ff700 1 -- 172.21.15.23:0/3200338266 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff394158a60 con 0x7ff3940c8a60 2023-03-27T10:02:10.774 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.775+0000 7ff389ffb700 1 -- 172.21.15.23:0/3200338266 <== mon.0 v2:172.21.15.23:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff394158a60 con 0x7ff3940c8a60 2023-03-27T10:02:10.778 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.777+0000 7ff39b7ff700 1 -- 172.21.15.23:0/3200338266 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7ff378066e30 msgr2=0x7ff3780692e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:10.778 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.777+0000 7ff39b7ff700 1 --2- 172.21.15.23:0/3200338266 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7ff378066e30 0x7ff3780692e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff394157180 tx=0x7ff384007de0 comp rx=0 tx=0).stop 2023-03-27T10:02:10.778 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.777+0000 7ff39b7ff700 1 -- 172.21.15.23:0/3200338266 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7ff3940c8a60 msgr2=0x7ff3940cf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:10.779 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.777+0000 7ff39b7ff700 1 --2- 172.21.15.23:0/3200338266 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7ff3940c8a60 0x7ff3940cf0b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff39000e8a0 tx=0x7ff39000ebb0 comp rx=0 tx=0).stop 2023-03-27T10:02:10.779 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.778+0000 7ff39b7ff700 1 -- 172.21.15.23:0/3200338266 shutdown_connections 2023-03-27T10:02:10.779 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.778+0000 7ff39b7ff700 1 --2- 172.21.15.23:0/3200338266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff39415a5f0 0x7ff3940b2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:10.780 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.778+0000 7ff39b7ff700 1 --2- 172.21.15.23:0/3200338266 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7ff3940c8a60 0x7ff3940cf0b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:10.780 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.778+0000 7ff39b7ff700 1 --2- 172.21.15.23:0/3200338266 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7ff378066e30 0x7ff3780692e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:10.780 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.778+0000 7ff39b7ff700 1 --2- 172.21.15.23:0/3200338266 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff3940c94b0 0x7ff3940cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:10.780 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.778+0000 7ff39b7ff700 1 -- 172.21.15.23:0/3200338266 >> 172.21.15.23:0/3200338266 conn(0x7ff39406be90 msgr2=0x7ff39406ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:10.781 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.778+0000 7ff39b7ff700 1 -- 172.21.15.23:0/3200338266 shutdown_connections 2023-03-27T10:02:10.781 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.778+0000 7ff39b7ff700 1 -- 172.21.15.23:0/3200338266 wait complete. 2023-03-27T10:02:10.782 INFO:teuthology.orchestra.run.smithi023.stderr:pool 'cephfs_metadata' created 2023-03-27T10:02:10.800 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-03-27T10:02:10.995 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.996+0000 7f40eb9ff700 1 Processor -- start 2023-03-27T10:02:10.996 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.996+0000 7f40eb9ff700 1 -- start start 2023-03-27T10:02:10.999 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.997+0000 7f40eb9ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f40e41509c0 0x7f40e40c5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:11.000 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.997+0000 7f40eb9ff700 1 -- --> v1:172.21.15.23:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40e41566d0 con 0x7f40e40c2da0 2023-03-27T10:02:11.000 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.997+0000 7f40eb9ff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40e4150d90 con 0x7f40e40ca7a0 2023-03-27T10:02:11.000 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.997+0000 7f40eb9ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40e40c4a70 con 0x7f40e41509c0 2023-03-27T10:02:11.001 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.997+0000 7f40e8f9a700 1 --1- >> v1:172.21.15.23:6789/0 conn(0x7f40e40c2da0 0x7f40e4157bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.23:6789/0 says I am v1:172.21.15.23:45846/0 (socket says 172.21.15.23:45846) 2023-03-27T10:02:11.001 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.997+0000 7f40e8f9a700 1 -- 172.21.15.23:0/931155852 learned_addr learned my addr 172.21.15.23:0/931155852 (peer_addr_for_me v1:172.21.15.23:0/0) 2023-03-27T10:02:11.001 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.998+0000 7f40e979b700 1 --2- 172.21.15.23:0/931155852 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f40e41509c0 0x7f40e40c5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:11.001 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.998+0000 7f40dbfff700 1 -- 172.21.15.23:0/931155852 <== mon.0 v1:172.21.15.23:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 760015346 0 0) 0x7f40e41566d0 con 0x7f40e40c2da0 2023-03-27T10:02:11.001 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.998+0000 7f40dbfff700 1 -- 172.21.15.23:0/931155852 --> v1:172.21.15.23:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40d00035b0 con 0x7f40e40c2da0 2023-03-27T10:02:11.001 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.998+0000 7f40dbfff700 1 -- 172.21.15.23:0/931155852 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3247409680 0 0) 0x7f40e4150d90 con 0x7f40e40ca7a0 2023-03-27T10:02:11.002 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.998+0000 7f40dbfff700 1 -- 172.21.15.23:0/931155852 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40e41566d0 con 0x7f40e40ca7a0 2023-03-27T10:02:11.002 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.998+0000 7f40e979b700 1 -- 172.21.15.23:0/931155852 >> v1:172.21.15.112:6789/0 conn(0x7f40e40ca7a0 legacy=0x7f40e4161050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:11.002 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.998+0000 7f40e979b700 1 -- 172.21.15.23:0/931155852 >> v1:172.21.15.23:6789/0 conn(0x7f40e40c2da0 legacy=0x7f40e4157bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:11.002 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.998+0000 7f40e979b700 1 -- 172.21.15.23:0/931155852 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e4156990 con 0x7f40e41509c0 2023-03-27T10:02:11.002 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.998+0000 7f40e979b700 1 --2- 172.21.15.23:0/931155852 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f40e41509c0 0x7f40e40c5190 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f40e0009870 tx=0x7f40e0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5249f8e923c2895b server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:11.002 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.998+0000 7f40dbfff700 1 -- 172.21.15.23:0/931155852 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e0011830 con 0x7f40e41509c0 2023-03-27T10:02:11.003 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.999+0000 7f40dbfff700 1 -- 172.21.15.23:0/931155852 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e0011990 con 0x7f40e41509c0 2023-03-27T10:02:11.003 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.999+0000 7f40dbfff700 1 -- 172.21.15.23:0/931155852 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e0011d20 con 0x7f40e41509c0 2023-03-27T10:02:11.003 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.999+0000 7f40eb9ff700 1 -- 172.21.15.23:0/931155852 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f40e41509c0 msgr2=0x7f40e40c5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:11.003 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.999+0000 7f40eb9ff700 1 --2- 172.21.15.23:0/931155852 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f40e41509c0 0x7f40e40c5190 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f40e0009870 tx=0x7f40e0009b80 comp rx=0 tx=0).stop 2023-03-27T10:02:11.003 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.999+0000 7f40eb9ff700 1 -- 172.21.15.23:0/931155852 shutdown_connections 2023-03-27T10:02:11.004 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.999+0000 7f40eb9ff700 1 --2- 172.21.15.23:0/931155852 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f40e41509c0 0x7f40e40c5190 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:11.004 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.999+0000 7f40eb9ff700 1 -- 172.21.15.23:0/931155852 >> 172.21.15.23:0/931155852 conn(0x7f40e406a1f0 msgr2=0x7f40e406a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:11.004 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.999+0000 7f40eb9ff700 1 -- 172.21.15.23:0/931155852 shutdown_connections 2023-03-27T10:02:11.004 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:10.999+0000 7f40eb9ff700 1 -- 172.21.15.23:0/931155852 wait complete. 2023-03-27T10:02:11.010 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.001+0000 7f40eb9ff700 1 Processor -- start 2023-03-27T10:02:11.010 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.001+0000 7f40eb9ff700 1 -- start start 2023-03-27T10:02:11.010 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.002+0000 7f40eb9ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f40e40c2da0 0x7f40e40c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:11.011 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.002+0000 7f40eb9ff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f40e40ca7a0 0x7f40e4070320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:11.011 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.002+0000 7f40eb9ff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40e41509c0 0x7f40e41456f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:11.011 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.002+0000 7f40eb9ff700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e40c7bc0 con 0x7f40e40ca7a0 2023-03-27T10:02:11.011 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.002+0000 7f40eb9ff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e40c7210 con 0x7f40e41509c0 2023-03-27T10:02:11.011 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.002+0000 7f40eb9ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40e4156150 con 0x7f40e40c2da0 2023-03-27T10:02:11.011 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.002+0000 7f40e8f9a700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f40e40ca7a0 0x7f40e4070320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:11.011 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.002+0000 7f40e8f9a700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f40e40ca7a0 0x7f40e4070320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:48724/0 (socket says 172.21.15.23:48724) 2023-03-27T10:02:11.012 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.002+0000 7f40e8f9a700 1 -- 172.21.15.23:0/2165087546 learned_addr learned my addr 172.21.15.23:0/2165087546 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:11.012 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.002+0000 7f40e979b700 1 --2- 172.21.15.23:0/2165087546 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f40e40c2da0 0x7f40e40c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:11.012 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.002+0000 7f40e9f9c700 1 --2- 172.21.15.23:0/2165087546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40e41509c0 0x7f40e41456f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:11.012 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.003+0000 7f40e8f9a700 1 -- 172.21.15.23:0/2165087546 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f40e40c2da0 msgr2=0x7f40e40c6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:11.012 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.003+0000 7f40e8f9a700 1 --2- 172.21.15.23:0/2165087546 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f40e40c2da0 0x7f40e40c6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:11.012 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.003+0000 7f40e8f9a700 1 -- 172.21.15.23:0/2165087546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40e41509c0 msgr2=0x7f40e41456f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:11.012 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.003+0000 7f40e8f9a700 1 --2- 172.21.15.23:0/2165087546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40e41509c0 0x7f40e41456f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:11.013 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.003+0000 7f40e8f9a700 1 -- 172.21.15.23:0/2165087546 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e00094a0 con 0x7f40e40ca7a0 2023-03-27T10:02:11.013 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.003+0000 7f40e8f9a700 1 --2- 172.21.15.23:0/2165087546 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f40e40ca7a0 0x7f40e4070320 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f40d4001eb0 tx=0x7f40d400c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:11.013 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.003+0000 7f40d9ffb700 1 -- 172.21.15.23:0/2165087546 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d4005d70 con 0x7f40e40ca7a0 2023-03-27T10:02:11.013 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.003+0000 7f40d9ffb700 1 -- 172.21.15.23:0/2165087546 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d40070d0 con 0x7f40e40ca7a0 2023-03-27T10:02:11.013 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.003+0000 7f40d9ffb700 1 -- 172.21.15.23:0/2165087546 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d4007440 con 0x7f40e40ca7a0 2023-03-27T10:02:11.013 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.003+0000 7f40eb9ff700 1 -- 172.21.15.23:0/2165087546 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40e4156f50 con 0x7f40e40ca7a0 2023-03-27T10:02:11.013 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.003+0000 7f40eb9ff700 1 -- 172.21.15.23:0/2165087546 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40e4156c70 con 0x7f40e40ca7a0 2023-03-27T10:02:11.014 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.004+0000 7f40eb9ff700 1 -- 172.21.15.23:0/2165087546 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40e4157230 con 0x7f40e40ca7a0 2023-03-27T10:02:11.014 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.005+0000 7f40d9ffb700 1 -- 172.21.15.23:0/2165087546 <== mon.0 v2:172.21.15.23:3300/0 4 ==== mgrmap(e 6) v1 ==== 85113+0+0 (secure 0 0 0) 0x7f40d4019030 con 0x7f40e40ca7a0 2023-03-27T10:02:11.014 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.006+0000 7f40d9ffb700 1 --2- 172.21.15.23:0/2165087546 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f40c8066e60 0x7f40c8069310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:11.014 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.006+0000 7f40d9ffb700 1 -- 172.21.15.23:0/2165087546 <== mon.0 v2:172.21.15.23:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f40d4083230 con 0x7f40e40ca7a0 2023-03-27T10:02:11.014 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.007+0000 7f40e979b700 1 --2- 172.21.15.23:0/2165087546 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f40c8066e60 0x7f40c8069310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:11.014 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.008+0000 7f40e979b700 1 --2- 172.21.15.23:0/2165087546 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f40c8066e60 0x7f40c8069310 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f40e00090b0 tx=0x7f40e001d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:11.014 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.011+0000 7f40d9ffb700 1 -- 172.21.15.23:0/2165087546 <== mon.0 v2:172.21.15.23:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173280 (secure 0 0 0) 0x7f40d404b4f0 con 0x7f40e40ca7a0 2023-03-27T10:02:11.214 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.214+0000 7f40eb9ff700 1 -- 172.21.15.23:0/2165087546 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f40e4069220 con 0x7f40e40ca7a0 2023-03-27T10:02:11.783 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.783+0000 7f40d9ffb700 1 -- 172.21.15.23:0/2165087546 <== mon.0 v2:172.21.15.23:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) v1 ==== 171+0+0 (secure 0 0 0) 0x7f40d4050680 con 0x7f40e40ca7a0 2023-03-27T10:02:11.786 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.787+0000 7f40eb9ff700 1 -- 172.21.15.23:0/2165087546 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f40c8066e60 msgr2=0x7f40c8069310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:11.786 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.787+0000 7f40eb9ff700 1 --2- 172.21.15.23:0/2165087546 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f40c8066e60 0x7f40c8069310 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f40e00090b0 tx=0x7f40e001d040 comp rx=0 tx=0).stop 2023-03-27T10:02:11.787 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.787+0000 7f40eb9ff700 1 -- 172.21.15.23:0/2165087546 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f40e40ca7a0 msgr2=0x7f40e4070320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:11.787 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.787+0000 7f40eb9ff700 1 --2- 172.21.15.23:0/2165087546 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f40e40ca7a0 0x7f40e4070320 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f40d4001eb0 tx=0x7f40d400c7b0 comp rx=0 tx=0).stop 2023-03-27T10:02:11.787 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.788+0000 7f40eb9ff700 1 -- 172.21.15.23:0/2165087546 shutdown_connections 2023-03-27T10:02:11.788 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.788+0000 7f40eb9ff700 1 --2- 172.21.15.23:0/2165087546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40e41509c0 0x7f40e41456f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:11.788 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.788+0000 7f40eb9ff700 1 --2- 172.21.15.23:0/2165087546 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f40c8066e60 0x7f40c8069310 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:11.788 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.788+0000 7f40eb9ff700 1 --2- 172.21.15.23:0/2165087546 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f40e40c2da0 0x7f40e40c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:11.788 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.788+0000 7f40eb9ff700 1 --2- 172.21.15.23:0/2165087546 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f40e40ca7a0 0x7f40e4070320 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:11.788 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.788+0000 7f40eb9ff700 1 -- 172.21.15.23:0/2165087546 >> 172.21.15.23:0/2165087546 conn(0x7f40e406a1f0 msgr2=0x7f40e406e740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:11.788 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.788+0000 7f40eb9ff700 1 -- 172.21.15.23:0/2165087546 shutdown_connections 2023-03-27T10:02:11.792 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:11.789+0000 7f40eb9ff700 1 -- 172.21.15.23:0/2165087546 wait complete. 2023-03-27T10:02:11.807 DEBUG:teuthology.orchestra.run.smithi023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-27T10:02:11.814 INFO:teuthology.orchestra.run.smithi023.stderr:pool 'cephfs_data' created 2023-03-27T10:02:11.818 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-27T10:02:11.829 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-03-27T10:02:12.061 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.061+0000 7f219a7ff700 1 Processor -- start 2023-03-27T10:02:12.062 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.061+0000 7f219a7ff700 1 -- start start 2023-03-27T10:02:12.065 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.062+0000 7f219a7ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f21940c7320 0x7f219406f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:12.065 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.062+0000 7f219a7ff700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f219406b300 0x7f219406b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:12.065 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.062+0000 7f219a7ff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f219406e820 con 0x7f21940c6920 2023-03-27T10:02:12.065 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.062+0000 7f219a7ff700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21940c86a0 con 0x7f219406b300 2023-03-27T10:02:12.066 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.062+0000 7f219a7ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2194159690 con 0x7f21940c7320 2023-03-27T10:02:12.066 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.063+0000 7f2198d9c700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f219406b300 0x7f219406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:12.066 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.063+0000 7f21937fe700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f21940c7320 0x7f219406f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:12.066 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.063+0000 7f2193fff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f21940c6920 0x7f21940c7f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.23:57280/0 (socket says 172.21.15.23:57280) 2023-03-27T10:02:12.066 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.063+0000 7f2193fff700 1 -- 172.21.15.23:0/3775393165 learned_addr learned my addr 172.21.15.23:0/3775393165 (peer_addr_for_me v1:172.21.15.23:0/0) 2023-03-27T10:02:12.067 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.063+0000 7f2198d9c700 1 -- 172.21.15.23:0/3775393165 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f21940c7320 msgr2=0x7f219406f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:02:12.067 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.063+0000 7f2192ffd700 1 -- 172.21.15.23:0/3775393165 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2843761369 0 0) 0x7f219406e820 con 0x7f21940c6920 2023-03-27T10:02:12.067 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.063+0000 7f2198d9c700 1 --2- 172.21.15.23:0/3775393165 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f21940c7320 0x7f219406f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:12.067 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.063+0000 7f2198d9c700 1 -- 172.21.15.23:0/3775393165 >> v1:172.21.15.112:6789/0 conn(0x7f21940c6920 legacy=0x7f21940c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:12.067 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.063+0000 7f2198d9c700 1 -- 172.21.15.23:0/3775393165 --> v2:172.21.15.23:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f219406dc10 con 0x7f219406b300 2023-03-27T10:02:12.067 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.063+0000 7f21937fe700 1 --2- 172.21.15.23:0/3775393165 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f21940c7320 0x7f219406f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-27T10:02:12.067 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.063+0000 7f2198d9c700 1 --2- 172.21.15.23:0/3775393165 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f219406b300 0x7f219406b760 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f218c009d90 tx=0x7f218c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d787b782c1e6c4 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:12.068 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.064+0000 7f2192ffd700 1 -- 172.21.15.23:0/3775393165 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218c00e070 con 0x7f219406b300 2023-03-27T10:02:12.068 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.064+0000 7f2192ffd700 1 -- 172.21.15.23:0/3775393165 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f218c00bb20 con 0x7f219406b300 2023-03-27T10:02:12.068 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.064+0000 7f2192ffd700 1 -- 172.21.15.23:0/3775393165 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218c012430 con 0x7f219406b300 2023-03-27T10:02:12.068 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.064+0000 7f219a7ff700 1 -- 172.21.15.23:0/3775393165 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f219406b300 msgr2=0x7f219406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:12.068 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.064+0000 7f219a7ff700 1 --2- 172.21.15.23:0/3775393165 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f219406b300 0x7f219406b760 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f218c009d90 tx=0x7f218c00b330 comp rx=0 tx=0).stop 2023-03-27T10:02:12.068 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.064+0000 7f219a7ff700 1 -- 172.21.15.23:0/3775393165 shutdown_connections 2023-03-27T10:02:12.068 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.064+0000 7f219a7ff700 1 --2- 172.21.15.23:0/3775393165 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f219406b300 0x7f219406b760 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:12.069 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.064+0000 7f219a7ff700 1 --2- 172.21.15.23:0/3775393165 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f21940c7320 0x7f219406f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:12.069 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.064+0000 7f219a7ff700 1 -- 172.21.15.23:0/3775393165 >> 172.21.15.23:0/3775393165 conn(0x7f2194065ca0 msgr2=0x7f2194066070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:12.069 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.064+0000 7f219a7ff700 1 -- 172.21.15.23:0/3775393165 shutdown_connections 2023-03-27T10:02:12.069 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.064+0000 7f219a7ff700 1 -- 172.21.15.23:0/3775393165 wait complete. 2023-03-27T10:02:12.071 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.066+0000 7f219a7ff700 1 Processor -- start 2023-03-27T10:02:12.071 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.066+0000 7f219a7ff700 1 -- start start 2023-03-27T10:02:12.071 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.067+0000 7f219a7ff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f219406b300 0x7f21940ca780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:12.071 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.067+0000 7f219a7ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f21940c6920 0x7f21940cca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:12.071 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.067+0000 7f219a7ff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21940c7320 0x7f21940c9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:12.072 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.067+0000 7f219a7ff700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2194159490 con 0x7f219406b300 2023-03-27T10:02:12.072 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.067+0000 7f219a7ff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2194158720 con 0x7f21940c7320 2023-03-27T10:02:12.072 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.067+0000 7f219a7ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21940cc580 con 0x7f21940c6920 2023-03-27T10:02:12.072 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.067+0000 7f2193fff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f219406b300 0x7f21940ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:12.072 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.067+0000 7f2193fff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f219406b300 0x7f21940ca780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:48738/0 (socket says 172.21.15.23:48738) 2023-03-27T10:02:12.072 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.067+0000 7f2193fff700 1 -- 172.21.15.23:0/3405770073 learned_addr learned my addr 172.21.15.23:0/3405770073 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:12.072 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.067+0000 7f21937fe700 1 --2- 172.21.15.23:0/3405770073 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f21940c6920 0x7f21940cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:12.073 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.067+0000 7f2198d9c700 1 --2- 172.21.15.23:0/3405770073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21940c7320 0x7f21940c9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:12.073 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.068+0000 7f2193fff700 1 -- 172.21.15.23:0/3405770073 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f21940c6920 msgr2=0x7f21940cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:12.073 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.068+0000 7f2193fff700 1 --2- 172.21.15.23:0/3405770073 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f21940c6920 0x7f21940cca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:12.073 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.068+0000 7f2193fff700 1 -- 172.21.15.23:0/3405770073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21940c7320 msgr2=0x7f21940c9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:12.073 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.068+0000 7f2193fff700 1 --2- 172.21.15.23:0/3405770073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21940c7320 0x7f21940c9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:12.073 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.068+0000 7f2193fff700 1 -- 172.21.15.23:0/3405770073 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21880094a0 con 0x7f219406b300 2023-03-27T10:02:12.073 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.068+0000 7f2193fff700 1 --2- 172.21.15.23:0/3405770073 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f219406b300 0x7f21940ca780 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f218400c780 tx=0x7f218400ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:12.074 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.068+0000 7f2190ff9700 1 -- 172.21.15.23:0/3405770073 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2184004c80 con 0x7f219406b300 2023-03-27T10:02:12.074 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.068+0000 7f2190ff9700 1 -- 172.21.15.23:0/3405770073 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2184004e00 con 0x7f219406b300 2023-03-27T10:02:12.074 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.068+0000 7f2190ff9700 1 -- 172.21.15.23:0/3405770073 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2184007310 con 0x7f219406b300 2023-03-27T10:02:12.074 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.068+0000 7f219a7ff700 1 -- 172.21.15.23:0/3405770073 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f218c009a70 con 0x7f219406b300 2023-03-27T10:02:12.074 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.068+0000 7f219a7ff700 1 -- 172.21.15.23:0/3405770073 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f219406e1d0 con 0x7f219406b300 2023-03-27T10:02:12.074 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.069+0000 7f219a7ff700 1 -- 172.21.15.23:0/3405770073 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f219406def0 con 0x7f219406b300 2023-03-27T10:02:12.074 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.070+0000 7f2190ff9700 1 -- 172.21.15.23:0/3405770073 <== mon.0 v2:172.21.15.23:3300/0 4 ==== mgrmap(e 6) v1 ==== 85113+0+0 (secure 0 0 0) 0x7f218401a020 con 0x7f219406b300 2023-03-27T10:02:12.074 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.071+0000 7f2190ff9700 1 --2- 172.21.15.23:0/3405770073 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f2180066e30 0x7f21800692e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:12.075 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.071+0000 7f2190ff9700 1 -- 172.21.15.23:0/3405770073 <== mon.0 v2:172.21.15.23:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f2184083d70 con 0x7f219406b300 2023-03-27T10:02:12.075 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.072+0000 7f21937fe700 1 --2- 172.21.15.23:0/3405770073 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f2180066e30 0x7f21800692e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:12.075 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.072+0000 7f21937fe700 1 --2- 172.21.15.23:0/3405770073 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f2180066e30 0x7f21800692e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f21941eb970 tx=0x7f2188009210 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:12.075 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.076+0000 7f2190ff9700 1 -- 172.21.15.23:0/3405770073 <== mon.0 v2:172.21.15.23:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173280 (secure 0 0 0) 0x7f2184050fb0 con 0x7f219406b300 2023-03-27T10:02:12.323 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.323+0000 7f219a7ff700 1 -- 172.21.15.23:0/3405770073 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f219406e4b0 con 0x7f219406b300 2023-03-27T10:02:12.330 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.329+0000 7f2190ff9700 1 -- 172.21.15.23:0/3405770073 <== mon.0 v2:172.21.15.23:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f21840545d0 con 0x7f219406b300 2023-03-27T10:02:12.335 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.335+0000 7f219a7ff700 1 -- 172.21.15.23:0/3405770073 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f2180066e30 msgr2=0x7f21800692e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:12.335 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.335+0000 7f219a7ff700 1 --2- 172.21.15.23:0/3405770073 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f2180066e30 0x7f21800692e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f21941eb970 tx=0x7f2188009210 comp rx=0 tx=0).stop 2023-03-27T10:02:12.336 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.335+0000 7f219a7ff700 1 -- 172.21.15.23:0/3405770073 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f219406b300 msgr2=0x7f21940ca780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:12.336 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.335+0000 7f219a7ff700 1 --2- 172.21.15.23:0/3405770073 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f219406b300 0x7f21940ca780 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f218400c780 tx=0x7f218400ca90 comp rx=0 tx=0).stop 2023-03-27T10:02:12.337 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.335+0000 7f219a7ff700 1 -- 172.21.15.23:0/3405770073 shutdown_connections 2023-03-27T10:02:12.337 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.335+0000 7f219a7ff700 1 --2- 172.21.15.23:0/3405770073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21940c7320 0x7f21940c9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:12.337 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.335+0000 7f219a7ff700 1 --2- 172.21.15.23:0/3405770073 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f219406b300 0x7f21940ca780 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:12.337 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.335+0000 7f219a7ff700 1 --2- 172.21.15.23:0/3405770073 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f2180066e30 0x7f21800692e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:12.337 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.335+0000 7f219a7ff700 1 --2- 172.21.15.23:0/3405770073 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f21940c6920 0x7f21940cca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:12.338 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.335+0000 7f219a7ff700 1 -- 172.21.15.23:0/3405770073 >> 172.21.15.23:0/3405770073 conn(0x7f2194065ca0 msgr2=0x7f21940c5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:12.338 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.336+0000 7f219a7ff700 1 -- 172.21.15.23:0/3405770073 shutdown_connections 2023-03-27T10:02:12.338 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.336+0000 7f219a7ff700 1 -- 172.21.15.23:0/3405770073 wait complete. 2023-03-27T10:02:12.348 INFO:teuthology.orchestra.run.smithi023.stderr:new fs with metadata pool 3 and data pool 4 2023-03-27T10:02:12.365 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-03-27T10:02:12.365 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-03-27T10:02:12.557 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.557+0000 7f32a39ff700 1 Processor -- start 2023-03-27T10:02:12.557 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.557+0000 7f32a39ff700 1 -- start start 2023-03-27T10:02:12.558 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.559+0000 7f32a39ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f329c0c94b0 0x7f329c06ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:12.558 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.559+0000 7f32a39ff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f329c06b360 0x7f329c15a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:12.559 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.559+0000 7f32a39ff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f329c158f30 con 0x7f329c0c8a60 2023-03-27T10:02:12.559 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.559+0000 7f32a39ff700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f329c15a1d0 con 0x7f329c06b360 2023-03-27T10:02:12.559 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.559+0000 7f32a39ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f329c159210 con 0x7f329c0c94b0 2023-03-27T10:02:12.559 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.559+0000 7f32a1f9c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f329c06b360 0x7f329c15a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:12.559 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.559+0000 7f32a0f9a700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f329c0c94b0 0x7f329c06ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:12.559 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.559+0000 7f32a179b700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f329c0c8a60 0x7f329c158760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.23:57294/0 (socket says 172.21.15.23:57294) 2023-03-27T10:02:12.559 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.559+0000 7f32a0f9a700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f329c0c94b0 0x7f329c06ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.23:49110/0 (socket says 172.21.15.23:49110) 2023-03-27T10:02:12.560 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.559+0000 7f32a0f9a700 1 -- 172.21.15.23:0/3122343510 learned_addr learned my addr 172.21.15.23:0/3122343510 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:12.560 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.559+0000 7f32a1f9c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f329c06b360 0x7f329c15a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.23:33874/0 (socket says 172.21.15.23:33874) 2023-03-27T10:02:12.561 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.560+0000 7f32a0f9a700 1 -- 172.21.15.23:0/3122343510 >> v2:172.21.15.112:3300/0 conn(0x7f329c06b360 msgr2=0x7f329c15a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:02:12.561 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.560+0000 7f32a0f9a700 1 --2- 172.21.15.23:0/3122343510 >> v2:172.21.15.112:3300/0 conn(0x7f329c06b360 0x7f329c15a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:12.561 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.560+0000 7f32a0f9a700 1 -- 172.21.15.23:0/3122343510 >> v1:172.21.15.112:6789/0 conn(0x7f329c0c8a60 legacy=0x7f329c158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:12.562 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.560+0000 7f32a0f9a700 1 -- 172.21.15.23:0/3122343510 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f329c1594f0 con 0x7f329c0c94b0 2023-03-27T10:02:12.562 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.560+0000 7f3293fff700 1 -- 172.21.15.23:0/3122343510 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4288878683 0 0) 0x7f329c158f30 con 0x7f329c0c8a60 2023-03-27T10:02:12.562 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.560+0000 7f32a1f9c700 1 --2- 172.21.15.23:0/3122343510 >> v2:172.21.15.112:3300/0 conn(0x7f329c06b360 0x7f329c15a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-27T10:02:12.562 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.560+0000 7f32a0f9a700 1 --2- 172.21.15.23:0/3122343510 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f329c0c94b0 0x7f329c06ae20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f328c0098f0 tx=0x7f328c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f53074d92bd0e103 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:12.562 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.560+0000 7f3293fff700 1 -- 172.21.15.23:0/3122343510 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328c011830 con 0x7f329c0c94b0 2023-03-27T10:02:12.563 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.560+0000 7f3293fff700 1 -- 172.21.15.23:0/3122343510 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f328c011990 con 0x7f329c0c94b0 2023-03-27T10:02:12.563 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.560+0000 7f3293fff700 1 -- 172.21.15.23:0/3122343510 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328c011d20 con 0x7f329c0c94b0 2023-03-27T10:02:12.563 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.561+0000 7f32a39ff700 1 -- 172.21.15.23:0/3122343510 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f329c0c94b0 msgr2=0x7f329c06ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:12.563 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.561+0000 7f32a39ff700 1 --2- 172.21.15.23:0/3122343510 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f329c0c94b0 0x7f329c06ae20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f328c0098f0 tx=0x7f328c009c00 comp rx=0 tx=0).stop 2023-03-27T10:02:12.563 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.561+0000 7f32a39ff700 1 -- 172.21.15.23:0/3122343510 shutdown_connections 2023-03-27T10:02:12.563 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.561+0000 7f32a39ff700 1 --2- 172.21.15.23:0/3122343510 >> v2:172.21.15.112:3300/0 conn(0x7f329c06b360 0x7f329c15a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:12.563 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.561+0000 7f32a39ff700 1 --2- 172.21.15.23:0/3122343510 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f329c0c94b0 0x7f329c06ae20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:12.564 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.561+0000 7f32a39ff700 1 -- 172.21.15.23:0/3122343510 >> 172.21.15.23:0/3122343510 conn(0x7f329c06be90 msgr2=0x7f329c06c290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:12.564 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.561+0000 7f32a39ff700 1 -- 172.21.15.23:0/3122343510 shutdown_connections 2023-03-27T10:02:12.564 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.561+0000 7f32a39ff700 1 -- 172.21.15.23:0/3122343510 wait complete. 2023-03-27T10:02:12.566 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.562+0000 7f32a39ff700 1 Processor -- start 2023-03-27T10:02:12.566 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.563+0000 7f32a39ff700 1 -- start start 2023-03-27T10:02:12.566 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.563+0000 7f32a39ff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f329c06b360 0x7f329c0b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:12.566 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.563+0000 7f32a39ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f329c0c8a60 0x7f329c0ce610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:12.567 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.563+0000 7f32a39ff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f329c0c94b0 0x7f329c15a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:12.567 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.563+0000 7f32a39ff700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329c0ceb50 con 0x7f329c06b360 2023-03-27T10:02:12.567 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.563+0000 7f32a39ff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329c1f21e0 con 0x7f329c0c94b0 2023-03-27T10:02:12.567 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.563+0000 7f32a39ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f329c0ce330 con 0x7f329c0c8a60 2023-03-27T10:02:12.567 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.564+0000 7f32a179b700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f329c06b360 0x7f329c0b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:12.567 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.564+0000 7f32a179b700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f329c06b360 0x7f329c0b46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:48752/0 (socket says 172.21.15.23:48752) 2023-03-27T10:02:12.568 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.564+0000 7f32a179b700 1 -- 172.21.15.23:0/1835453563 learned_addr learned my addr 172.21.15.23:0/1835453563 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:12.568 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.564+0000 7f32a1f9c700 1 --2- 172.21.15.23:0/1835453563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f329c0c94b0 0x7f329c15a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:12.568 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.564+0000 7f32a0f9a700 1 --2- 172.21.15.23:0/1835453563 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f329c0c8a60 0x7f329c0ce610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:12.568 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.564+0000 7f32a179b700 1 -- 172.21.15.23:0/1835453563 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f329c0c8a60 msgr2=0x7f329c0ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:12.568 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.564+0000 7f32a179b700 1 --2- 172.21.15.23:0/1835453563 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f329c0c8a60 0x7f329c0ce610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:12.568 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.564+0000 7f32a179b700 1 -- 172.21.15.23:0/1835453563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f329c0c94b0 msgr2=0x7f329c15a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:12.568 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.564+0000 7f32a179b700 1 --2- 172.21.15.23:0/1835453563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f329c0c94b0 0x7f329c15a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:12.569 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.564+0000 7f32a179b700 1 -- 172.21.15.23:0/1835453563 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3294009a70 con 0x7f329c06b360 2023-03-27T10:02:12.569 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.564+0000 7f32a179b700 1 --2- 172.21.15.23:0/1835453563 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f329c06b360 0x7f329c0b46e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f329800c750 tx=0x7f329800ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:12.569 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.565+0000 7f3291ffb700 1 -- 172.21.15.23:0/1835453563 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3298004c80 con 0x7f329c06b360 2023-03-27T10:02:12.569 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.565+0000 7f3291ffb700 1 -- 172.21.15.23:0/1835453563 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3298004e00 con 0x7f329c06b360 2023-03-27T10:02:12.569 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.565+0000 7f3291ffb700 1 -- 172.21.15.23:0/1835453563 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3298007310 con 0x7f329c06b360 2023-03-27T10:02:12.569 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.565+0000 7f32a39ff700 1 -- 172.21.15.23:0/1835453563 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f328c009550 con 0x7f329c06b360 2023-03-27T10:02:12.570 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.565+0000 7f32a39ff700 1 -- 172.21.15.23:0/1835453563 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f329c0c8e30 con 0x7f329c06b360 2023-03-27T10:02:12.570 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.566+0000 7f32a39ff700 1 -- 172.21.15.23:0/1835453563 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f329c159df0 con 0x7f329c06b360 2023-03-27T10:02:12.570 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.567+0000 7f3291ffb700 1 -- 172.21.15.23:0/1835453563 <== mon.0 v2:172.21.15.23:3300/0 4 ==== mgrmap(e 6) v1 ==== 85113+0+0 (secure 0 0 0) 0x7f329801a020 con 0x7f329c06b360 2023-03-27T10:02:12.571 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.568+0000 7f3291ffb700 1 --2- 172.21.15.23:0/1835453563 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f3280066e60 0x7f3280069310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:12.571 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.568+0000 7f3291ffb700 1 -- 172.21.15.23:0/1835453563 <== mon.0 v2:172.21.15.23:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3298083e10 con 0x7f329c06b360 2023-03-27T10:02:12.571 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.568+0000 7f32a0f9a700 1 --2- 172.21.15.23:0/1835453563 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f3280066e60 0x7f3280069310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:12.572 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.569+0000 7f32a0f9a700 1 --2- 172.21.15.23:0/1835453563 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f3280066e60 0x7f3280069310 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f328c000c00 tx=0x7f328c01d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:12.572 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.572+0000 7f3291ffb700 1 -- 172.21.15.23:0/1835453563 <== mon.0 v2:172.21.15.23:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173280 (secure 0 0 0) 0x7f3298087030 con 0x7f329c06b360 2023-03-27T10:02:12.784 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:12.785+0000 7f32a39ff700 1 -- 172.21.15.23:0/1835453563 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f329c0c4e70 con 0x7f329c06b360 2023-03-27T10:02:13.331 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.332+0000 7f3291ffb700 1 -- 172.21.15.23:0/1835453563 <== mon.0 v2:172.21.15.23:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) v1 ==== 157+0+0 (secure 0 0 0) 0x7f32980510a0 con 0x7f329c06b360 2023-03-27T10:02:13.334 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.334+0000 7f32a39ff700 1 -- 172.21.15.23:0/1835453563 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f3280066e60 msgr2=0x7f3280069310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:13.334 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.334+0000 7f32a39ff700 1 --2- 172.21.15.23:0/1835453563 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f3280066e60 0x7f3280069310 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f328c000c00 tx=0x7f328c01d040 comp rx=0 tx=0).stop 2023-03-27T10:02:13.335 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.334+0000 7f32a39ff700 1 -- 172.21.15.23:0/1835453563 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f329c06b360 msgr2=0x7f329c0b46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:13.335 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.334+0000 7f32a39ff700 1 --2- 172.21.15.23:0/1835453563 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f329c06b360 0x7f329c0b46e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f329800c750 tx=0x7f329800ca60 comp rx=0 tx=0).stop 2023-03-27T10:02:13.335 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.335+0000 7f32a39ff700 1 -- 172.21.15.23:0/1835453563 shutdown_connections 2023-03-27T10:02:13.335 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.335+0000 7f32a39ff700 1 --2- 172.21.15.23:0/1835453563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f329c0c94b0 0x7f329c15a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:13.335 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.335+0000 7f32a39ff700 1 --2- 172.21.15.23:0/1835453563 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f329c06b360 0x7f329c0b46e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:13.335 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.335+0000 7f32a39ff700 1 --2- 172.21.15.23:0/1835453563 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f3280066e60 0x7f3280069310 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:13.335 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.335+0000 7f32a39ff700 1 --2- 172.21.15.23:0/1835453563 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f329c0c8a60 0x7f329c0ce610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:13.336 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.335+0000 7f32a39ff700 1 -- 172.21.15.23:0/1835453563 >> 172.21.15.23:0/1835453563 conn(0x7f329c06be90 msgr2=0x7f329c0ca670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:13.336 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.335+0000 7f32a39ff700 1 -- 172.21.15.23:0/1835453563 shutdown_connections 2023-03-27T10:02:13.336 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.335+0000 7f32a39ff700 1 -- 172.21.15.23:0/1835453563 wait complete. 2023-03-27T10:02:13.355 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-03-27T10:02:13.556 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.556+0000 7fad713ff700 1 Processor -- start 2023-03-27T10:02:13.556 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.557+0000 7fad713ff700 1 -- start start 2023-03-27T10:02:13.557 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.558+0000 7fad713ff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fad6c152b70 0x7fad6c06b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:13.558 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.558+0000 7fad713ff700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7fad6c15c980 0x7fad6c0ca100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:13.558 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.558+0000 7fad713ff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad6c0c8f90 con 0x7fad6c0b4480 2023-03-27T10:02:13.558 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.558+0000 7fad713ff700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad6c06c750 con 0x7fad6c15c980 2023-03-27T10:02:13.558 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.558+0000 7fad713ff700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad6c0c9c70 con 0x7fad6c152b70 2023-03-27T10:02:13.558 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.558+0000 7fad6a7fc700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7fad6c15c980 0x7fad6c0ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:13.558 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.558+0000 7fad6a7fc700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7fad6c15c980 0x7fad6c0ca100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:48766/0 (socket says 172.21.15.23:48766) 2023-03-27T10:02:13.558 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.558+0000 7fad6a7fc700 1 -- 172.21.15.23:0/49385016 learned_addr learned my addr 172.21.15.23:0/49385016 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:13.559 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.558+0000 7fad6affd700 1 --2- 172.21.15.23:0/49385016 >> v2:172.21.15.112:3300/0 conn(0x7fad6c152b70 0x7fad6c06b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:13.560 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.559+0000 7fad6a7fc700 1 -- 172.21.15.23:0/49385016 >> v2:172.21.15.112:3300/0 conn(0x7fad6c152b70 msgr2=0x7fad6c06b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:02:13.560 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.559+0000 7fad6a7fc700 1 --2- 172.21.15.23:0/49385016 >> v2:172.21.15.112:3300/0 conn(0x7fad6c152b70 0x7fad6c06b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:13.560 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.559+0000 7fad6a7fc700 1 -- 172.21.15.23:0/49385016 >> v1:172.21.15.112:6789/0 conn(0x7fad6c0b4480 legacy=0x7fad6c0c6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:13.560 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.559+0000 7fad6a7fc700 1 -- 172.21.15.23:0/49385016 --> v2:172.21.15.23:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad6c0c8760 con 0x7fad6c15c980 2023-03-27T10:02:13.560 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.559+0000 7fad6a7fc700 1 --2- 172.21.15.23:0/49385016 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fad6c15c980 0x7fad6c0ca100 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fad6c0cc680 tx=0x7fad60009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e83ca4ea2c4eac server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:13.561 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.559+0000 7fad69ffb700 1 -- 172.21.15.23:0/49385016 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad60011830 con 0x7fad6c15c980 2023-03-27T10:02:13.561 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.559+0000 7fad69ffb700 1 -- 172.21.15.23:0/49385016 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad60011990 con 0x7fad6c15c980 2023-03-27T10:02:13.561 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.559+0000 7fad69ffb700 1 -- 172.21.15.23:0/49385016 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad60011d20 con 0x7fad6c15c980 2023-03-27T10:02:13.561 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.560+0000 7fad713ff700 1 -- 172.21.15.23:0/49385016 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fad6c15c980 msgr2=0x7fad6c0ca100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:13.561 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.560+0000 7fad713ff700 1 --2- 172.21.15.23:0/49385016 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fad6c15c980 0x7fad6c0ca100 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fad6c0cc680 tx=0x7fad60009bd0 comp rx=0 tx=0).stop 2023-03-27T10:02:13.561 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.560+0000 7fad713ff700 1 -- 172.21.15.23:0/49385016 shutdown_connections 2023-03-27T10:02:13.561 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.560+0000 7fad713ff700 1 --2- 172.21.15.23:0/49385016 >> v2:172.21.15.112:3300/0 conn(0x7fad6c152b70 0x7fad6c06b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:13.562 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.560+0000 7fad713ff700 1 --2- 172.21.15.23:0/49385016 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fad6c15c980 0x7fad6c0ca100 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:13.562 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.560+0000 7fad713ff700 1 -- 172.21.15.23:0/49385016 >> 172.21.15.23:0/49385016 conn(0x7fad6c065ca0 msgr2=0x7fad6c0660a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:13.562 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.560+0000 7fad713ff700 1 -- 172.21.15.23:0/49385016 shutdown_connections 2023-03-27T10:02:13.562 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.560+0000 7fad713ff700 1 -- 172.21.15.23:0/49385016 wait complete. 2023-03-27T10:02:13.563 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.562+0000 7fad713ff700 1 Processor -- start 2023-03-27T10:02:13.563 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.562+0000 7fad713ff700 1 -- start start 2023-03-27T10:02:13.563 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.564+0000 7fad713ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fad6c0b4480 0x7fad6c1edee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:13.563 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.564+0000 7fad713ff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad6c152b70 0x7fad6c1ef3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:13.563 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.564+0000 7fad713ff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fad6c15c980 0x7fad6c0b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:13.563 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.564+0000 7fad713ff700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6c1ef910 con 0x7fad6c15c980 2023-03-27T10:02:13.564 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.564+0000 7fad713ff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6c0cd930 con 0x7fad6c152b70 2023-03-27T10:02:13.564 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.564+0000 7fad713ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad6c0b42d0 con 0x7fad6c0b4480 2023-03-27T10:02:13.565 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.564+0000 7fad6b7fe700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fad6c15c980 0x7fad6c0b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:13.565 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.564+0000 7fad6b7fe700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fad6c15c980 0x7fad6c0b3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:48776/0 (socket says 172.21.15.23:48776) 2023-03-27T10:02:13.565 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.564+0000 7fad6b7fe700 1 -- 172.21.15.23:0/318386493 learned_addr learned my addr 172.21.15.23:0/318386493 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:13.566 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.564+0000 7fad6affd700 1 --2- 172.21.15.23:0/318386493 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fad6c0b4480 0x7fad6c1edee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:13.566 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.564+0000 7fad6a7fc700 1 --2- 172.21.15.23:0/318386493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad6c152b70 0x7fad6c1ef3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:13.566 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.564+0000 7fad6b7fe700 1 -- 172.21.15.23:0/318386493 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fad6c0b4480 msgr2=0x7fad6c1edee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:13.566 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.564+0000 7fad6b7fe700 1 --2- 172.21.15.23:0/318386493 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fad6c0b4480 0x7fad6c1edee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:13.566 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.564+0000 7fad6b7fe700 1 -- 172.21.15.23:0/318386493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad6c152b70 msgr2=0x7fad6c1ef3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:13.566 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.564+0000 7fad6b7fe700 1 --2- 172.21.15.23:0/318386493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad6c152b70 0x7fad6c1ef3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:13.566 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.564+0000 7fad6b7fe700 1 -- 172.21.15.23:0/318386493 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad60009550 con 0x7fad6c15c980 2023-03-27T10:02:13.567 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.564+0000 7fad6affd700 1 --2- 172.21.15.23:0/318386493 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fad6c0b4480 0x7fad6c1edee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-27T10:02:13.567 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.565+0000 7fad6b7fe700 1 --2- 172.21.15.23:0/318386493 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fad6c15c980 0x7fad6c0b3d90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fad6400cc50 tx=0x7fad6400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:13.567 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.565+0000 7fad537fe700 1 -- 172.21.15.23:0/318386493 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad64003c00 con 0x7fad6c15c980 2023-03-27T10:02:13.567 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.565+0000 7fad537fe700 1 -- 172.21.15.23:0/318386493 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad64003d60 con 0x7fad6c15c980 2023-03-27T10:02:13.567 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.565+0000 7fad713ff700 1 -- 172.21.15.23:0/318386493 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad6c0c8cb0 con 0x7fad6c15c980 2023-03-27T10:02:13.567 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.565+0000 7fad537fe700 1 -- 172.21.15.23:0/318386493 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6400ab60 con 0x7fad6c15c980 2023-03-27T10:02:13.567 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.565+0000 7fad713ff700 1 -- 172.21.15.23:0/318386493 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad6c0c89d0 con 0x7fad6c15c980 2023-03-27T10:02:13.567 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.566+0000 7fad713ff700 1 -- 172.21.15.23:0/318386493 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad6c0c9620 con 0x7fad6c15c980 2023-03-27T10:02:13.568 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.567+0000 7fad537fe700 1 -- 172.21.15.23:0/318386493 <== mon.0 v2:172.21.15.23:3300/0 4 ==== mgrmap(e 6) v1 ==== 85113+0+0 (secure 0 0 0) 0x7fad64010050 con 0x7fad6c15c980 2023-03-27T10:02:13.568 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.568+0000 7fad537fe700 1 --2- 172.21.15.23:0/318386493 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fad58066e60 0x7fad58069310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:13.568 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.568+0000 7fad537fe700 1 -- 172.21.15.23:0/318386493 <== mon.0 v2:172.21.15.23:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6288+0+0 (secure 0 0 0) 0x7fad64084120 con 0x7fad6c15c980 2023-03-27T10:02:13.569 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.569+0000 7fad6affd700 1 --2- 172.21.15.23:0/318386493 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fad58066e60 0x7fad58069310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:13.569 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.569+0000 7fad6affd700 1 --2- 172.21.15.23:0/318386493 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fad58066e60 0x7fad58069310 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fad5c009c30 tx=0x7fad5c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:13.572 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.572+0000 7fad537fe700 1 -- 172.21.15.23:0/318386493 <== mon.0 v2:172.21.15.23:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173280 (secure 0 0 0) 0x7fad640512a0 con 0x7fad6c15c980 2023-03-27T10:02:13.775 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:13.775+0000 7fad713ff700 1 -- 172.21.15.23:0/318386493 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fad6c0c8f90 con 0x7fad6c15c980 2023-03-27T10:02:15.773 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:15.773+0000 7fad537fe700 1 -- 172.21.15.23:0/318386493 <== mon.0 v2:172.21.15.23:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) v1 ==== 228+0+0 (secure 0 0 0) 0x7fad6c0c8f90 con 0x7fad6c15c980 2023-03-27T10:02:15.781 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:15.781+0000 7fad713ff700 1 -- 172.21.15.23:0/318386493 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fad58066e60 msgr2=0x7fad58069310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:15.781 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:15.781+0000 7fad713ff700 1 --2- 172.21.15.23:0/318386493 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fad58066e60 0x7fad58069310 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fad5c009c30 tx=0x7fad5c008040 comp rx=0 tx=0).stop 2023-03-27T10:02:15.781 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:15.781+0000 7fad713ff700 1 -- 172.21.15.23:0/318386493 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fad6c15c980 msgr2=0x7fad6c0b3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:15.782 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:15.781+0000 7fad713ff700 1 --2- 172.21.15.23:0/318386493 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fad6c15c980 0x7fad6c0b3d90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fad6400cc50 tx=0x7fad6400cf60 comp rx=0 tx=0).stop 2023-03-27T10:02:15.782 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:15.783+0000 7fad713ff700 1 -- 172.21.15.23:0/318386493 shutdown_connections 2023-03-27T10:02:15.782 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:15.783+0000 7fad713ff700 1 --2- 172.21.15.23:0/318386493 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fad6c15c980 0x7fad6c0b3d90 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:15.782 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:15.783+0000 7fad713ff700 1 --2- 172.21.15.23:0/318386493 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fad58066e60 0x7fad58069310 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fad5c009c30 tx=0x7fad5c008040 comp rx=0 tx=0).stop 2023-03-27T10:02:15.782 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:15.783+0000 7fad713ff700 1 --2- 172.21.15.23:0/318386493 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fad6c0b4480 0x7fad6c1edee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:15.783 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:15.783+0000 7fad713ff700 1 --2- 172.21.15.23:0/318386493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad6c152b70 0x7fad6c1ef3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:15.783 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:15.783+0000 7fad713ff700 1 -- 172.21.15.23:0/318386493 >> 172.21.15.23:0/318386493 conn(0x7fad6c065ca0 msgr2=0x7fad6c158e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:15.783 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:15.783+0000 7fad713ff700 1 -- 172.21.15.23:0/318386493 shutdown_connections 2023-03-27T10:02:15.783 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:15.783+0000 7fad713ff700 1 -- 172.21.15.23:0/318386493 wait complete. 2023-03-27T10:02:15.787 INFO:teuthology.orchestra.run.smithi023.stderr:pool 'cephfs_data_ec' created 2023-03-27T10:02:15.806 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-03-27T10:02:16.003 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.003+0000 7f76a75ff700 1 Processor -- start 2023-03-27T10:02:16.004 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.003+0000 7f76a75ff700 1 -- start start 2023-03-27T10:02:16.007 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.004+0000 7f76a75ff700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f76a00c8a60 0x7f76a0158760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:16.007 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.005+0000 7f76a75ff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f76a015a5f0 0x7f76a015aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:16.007 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.005+0000 7f76a75ff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a0158ca0 con 0x7f76a00c94b0 2023-03-27T10:02:16.007 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.005+0000 7f76a75ff700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76a015a1d0 con 0x7f76a00c8a60 2023-03-27T10:02:16.008 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.005+0000 7f76a75ff700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76a0158f30 con 0x7f76a015a5f0 2023-03-27T10:02:16.008 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.005+0000 7f76a539b700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f76a00c8a60 0x7f76a0158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:16.008 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.005+0000 7f76a539b700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f76a00c8a60 0x7f76a0158760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:48788/0 (socket says 172.21.15.23:48788) 2023-03-27T10:02:16.008 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.005+0000 7f76a539b700 1 -- 172.21.15.23:0/4236010628 learned_addr learned my addr 172.21.15.23:0/4236010628 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:16.008 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.005+0000 7f76a5b9c700 1 --2- 172.21.15.23:0/4236010628 >> v2:172.21.15.112:3300/0 conn(0x7f76a015a5f0 0x7f76a015aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:16.008 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.005+0000 7f76a539b700 1 -- 172.21.15.23:0/4236010628 >> v2:172.21.15.112:3300/0 conn(0x7f76a015a5f0 msgr2=0x7f76a015aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:02:16.008 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.005+0000 7f76a539b700 1 --2- 172.21.15.23:0/4236010628 >> v2:172.21.15.112:3300/0 conn(0x7f76a015a5f0 0x7f76a015aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:16.008 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.005+0000 7f76a539b700 1 -- 172.21.15.23:0/4236010628 >> v1:172.21.15.112:6789/0 conn(0x7f76a00c94b0 legacy=0x7f76a006ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:16.009 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.005+0000 7f76a539b700 1 -- 172.21.15.23:0/4236010628 --> v2:172.21.15.23:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a0159210 con 0x7f76a00c8a60 2023-03-27T10:02:16.009 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.005+0000 7f76a539b700 1 --2- 172.21.15.23:0/4236010628 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f76a00c8a60 0x7f76a0158760 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f76a008c9e0 tx=0x7f769c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc285be9aab66fe6 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:16.009 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.006+0000 7f7697fff700 1 -- 172.21.15.23:0/4236010628 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c00b870 con 0x7f76a00c8a60 2023-03-27T10:02:16.009 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.006+0000 7f7697fff700 1 -- 172.21.15.23:0/4236010628 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769c00b9d0 con 0x7f76a00c8a60 2023-03-27T10:02:16.009 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.006+0000 7f7697fff700 1 -- 172.21.15.23:0/4236010628 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c00bd60 con 0x7f76a00c8a60 2023-03-27T10:02:16.010 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.006+0000 7f76a75ff700 1 -- 172.21.15.23:0/4236010628 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f76a00c8a60 msgr2=0x7f76a0158760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:16.010 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.006+0000 7f76a75ff700 1 --2- 172.21.15.23:0/4236010628 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f76a00c8a60 0x7f76a0158760 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f76a008c9e0 tx=0x7f769c009bd0 comp rx=0 tx=0).stop 2023-03-27T10:02:16.010 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.006+0000 7f76a75ff700 1 -- 172.21.15.23:0/4236010628 shutdown_connections 2023-03-27T10:02:16.010 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.006+0000 7f76a75ff700 1 --2- 172.21.15.23:0/4236010628 >> v2:172.21.15.112:3300/0 conn(0x7f76a015a5f0 0x7f76a015aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:16.010 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.006+0000 7f76a75ff700 1 --2- 172.21.15.23:0/4236010628 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f76a00c8a60 0x7f76a0158760 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:16.010 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.006+0000 7f76a75ff700 1 -- 172.21.15.23:0/4236010628 >> 172.21.15.23:0/4236010628 conn(0x7f76a006be90 msgr2=0x7f76a006c290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:16.010 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.006+0000 7f76a75ff700 1 -- 172.21.15.23:0/4236010628 shutdown_connections 2023-03-27T10:02:16.010 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.006+0000 7f76a75ff700 1 -- 172.21.15.23:0/4236010628 wait complete. 2023-03-27T10:02:16.012 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.008+0000 7f76a75ff700 1 Processor -- start 2023-03-27T10:02:16.012 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.008+0000 7f76a75ff700 1 -- start start 2023-03-27T10:02:16.012 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.009+0000 7f76a75ff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f76a00c8a60 0x7f76a01545a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:16.013 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.009+0000 7f76a75ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f76a00c94b0 0x7f76a006b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:16.013 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.009+0000 7f76a75ff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76a015a5f0 0x7f76a0155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:16.013 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.009+0000 7f76a75ff700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a00cc300 con 0x7f76a00c8a60 2023-03-27T10:02:16.013 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.009+0000 7f76a75ff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a0153b40 con 0x7f76a015a5f0 2023-03-27T10:02:16.013 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.009+0000 7f76a75ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a0150ef0 con 0x7f76a00c94b0 2023-03-27T10:02:16.013 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.009+0000 7f76a539b700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f76a00c8a60 0x7f76a01545a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:16.014 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.010+0000 7f76a539b700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f76a00c8a60 0x7f76a01545a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:48798/0 (socket says 172.21.15.23:48798) 2023-03-27T10:02:16.014 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.010+0000 7f76a539b700 1 -- 172.21.15.23:0/2415294348 learned_addr learned my addr 172.21.15.23:0/2415294348 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:16.014 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.010+0000 7f76a4b9a700 1 --2- 172.21.15.23:0/2415294348 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f76a00c94b0 0x7f76a006b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:16.014 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.010+0000 7f76a5b9c700 1 --2- 172.21.15.23:0/2415294348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76a015a5f0 0x7f76a0155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:16.014 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.010+0000 7f76a539b700 1 -- 172.21.15.23:0/2415294348 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f76a00c94b0 msgr2=0x7f76a006b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:16.014 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.010+0000 7f76a539b700 1 --2- 172.21.15.23:0/2415294348 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f76a00c94b0 0x7f76a006b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:16.015 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.010+0000 7f76a539b700 1 -- 172.21.15.23:0/2415294348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76a015a5f0 msgr2=0x7f76a0155600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:16.015 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.010+0000 7f76a539b700 1 --2- 172.21.15.23:0/2415294348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76a015a5f0 0x7f76a0155600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:16.015 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.010+0000 7f76a539b700 1 -- 172.21.15.23:0/2415294348 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f769c009550 con 0x7f76a00c8a60 2023-03-27T10:02:16.015 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.010+0000 7f76a539b700 1 --2- 172.21.15.23:0/2415294348 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f76a00c8a60 0x7f76a01545a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f76a006b9f0 tx=0x7f769c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:16.015 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.010+0000 7f7695ffb700 1 -- 172.21.15.23:0/2415294348 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c004310 con 0x7f76a00c8a60 2023-03-27T10:02:16.015 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.010+0000 7f7695ffb700 1 -- 172.21.15.23:0/2415294348 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769c01e070 con 0x7f76a00c8a60 2023-03-27T10:02:16.015 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.010+0000 7f76a75ff700 1 -- 172.21.15.23:0/2415294348 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a0159df0 con 0x7f76a00c8a60 2023-03-27T10:02:16.015 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.010+0000 7f7695ffb700 1 -- 172.21.15.23:0/2415294348 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c0036a0 con 0x7f76a00c8a60 2023-03-27T10:02:16.016 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.011+0000 7f76a75ff700 1 -- 172.21.15.23:0/2415294348 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a01594f0 con 0x7f76a00c8a60 2023-03-27T10:02:16.016 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.011+0000 7f76a75ff700 1 -- 172.21.15.23:0/2415294348 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a015af40 con 0x7f76a00c8a60 2023-03-27T10:02:16.016 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.012+0000 7f7695ffb700 1 -- 172.21.15.23:0/2415294348 <== mon.0 v2:172.21.15.23:3300/0 4 ==== mgrmap(e 6) v1 ==== 85113+0+0 (secure 0 0 0) 0x7f769c029020 con 0x7f76a00c8a60 2023-03-27T10:02:16.017 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.013+0000 7f7695ffb700 1 --2- 172.21.15.23:0/2415294348 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f7684066e30 0x7f76840692e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:16.017 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.013+0000 7f7695ffb700 1 -- 172.21.15.23:0/2415294348 <== mon.0 v2:172.21.15.23:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6781+0+0 (secure 0 0 0) 0x7f769c0855d0 con 0x7f76a00c8a60 2023-03-27T10:02:16.017 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.014+0000 7f76a4b9a700 1 --2- 172.21.15.23:0/2415294348 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f7684066e30 0x7f76840692e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:16.018 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.014+0000 7f76a4b9a700 1 --2- 172.21.15.23:0/2415294348 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f7684066e30 0x7f76840692e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f76a00c7fe0 tx=0x7f7690009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:16.018 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.018+0000 7f7695ffb700 1 -- 172.21.15.23:0/2415294348 <== mon.0 v2:172.21.15.23:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173280 (secure 0 0 0) 0x7f769c052540 con 0x7f76a00c8a60 2023-03-27T10:02:16.215 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.216+0000 7f76a75ff700 1 -- 172.21.15.23:0/2415294348 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f76a0155300 con 0x7f76a00c8a60 2023-03-27T10:02:16.774 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.774+0000 7f7695ffb700 1 -- 172.21.15.23:0/2415294348 <== mon.0 v2:172.21.15.23:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 5 allow_ec_overwrites to true v23) v1 ==== 169+0+0 (secure 0 0 0) 0x7f769c055b60 con 0x7f76a00c8a60 2023-03-27T10:02:16.777 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.778+0000 7f76a75ff700 1 -- 172.21.15.23:0/2415294348 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f7684066e30 msgr2=0x7f76840692e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:16.778 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.778+0000 7f76a75ff700 1 --2- 172.21.15.23:0/2415294348 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f7684066e30 0x7f76840692e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f76a00c7fe0 tx=0x7f7690009040 comp rx=0 tx=0).stop 2023-03-27T10:02:16.778 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.778+0000 7f76a75ff700 1 -- 172.21.15.23:0/2415294348 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f76a00c8a60 msgr2=0x7f76a01545a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:16.778 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.778+0000 7f76a75ff700 1 --2- 172.21.15.23:0/2415294348 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f76a00c8a60 0x7f76a01545a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f76a006b9f0 tx=0x7f769c004750 comp rx=0 tx=0).stop 2023-03-27T10:02:16.779 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.780+0000 7f76a75ff700 1 -- 172.21.15.23:0/2415294348 shutdown_connections 2023-03-27T10:02:16.779 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.780+0000 7f76a75ff700 1 --2- 172.21.15.23:0/2415294348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76a015a5f0 0x7f76a0155600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:16.780 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.780+0000 7f76a75ff700 1 --2- 172.21.15.23:0/2415294348 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f76a00c8a60 0x7f76a01545a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:16.780 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.780+0000 7f76a75ff700 1 --2- 172.21.15.23:0/2415294348 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f7684066e30 0x7f76840692e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:16.780 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.780+0000 7f76a75ff700 1 --2- 172.21.15.23:0/2415294348 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f76a00c94b0 0x7f76a006b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:16.780 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.780+0000 7f76a75ff700 1 -- 172.21.15.23:0/2415294348 >> 172.21.15.23:0/2415294348 conn(0x7f76a006be90 msgr2=0x7f76a0157730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:16.780 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.780+0000 7f76a75ff700 1 -- 172.21.15.23:0/2415294348 shutdown_connections 2023-03-27T10:02:16.781 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:16.781+0000 7f76a75ff700 1 -- 172.21.15.23:0/2415294348 wait complete. 2023-03-27T10:02:16.783 INFO:teuthology.orchestra.run.smithi023.stderr:set pool 5 allow_ec_overwrites to true 2023-03-27T10:02:16.816 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-03-27T10:02:17.007 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.008+0000 7fed925ff700 1 Processor -- start 2023-03-27T10:02:17.008 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.008+0000 7fed925ff700 1 -- start start 2023-03-27T10:02:17.009 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.009+0000 7fed925ff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fed8c0c5c90 0x7fed8c0c8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:17.009 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.009+0000 7fed925ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed8c070bb0 0x7fed8c0c6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:17.009 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.009+0000 7fed925ff700 1 -- --> v1:172.21.15.23:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed8c1592a0 con 0x7fed8c159730 2023-03-27T10:02:17.009 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.009+0000 7fed925ff700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed8c072310 con 0x7fed8c0c5c90 2023-03-27T10:02:17.009 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.009+0000 7fed925ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed8c158990 con 0x7fed8c070bb0 2023-03-27T10:02:17.009 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.009+0000 7fed8b7fe700 1 --1- >> v1:172.21.15.23:6789/0 conn(0x7fed8c159730 0x7fed8c06a1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.23:6789/0 says I am v1:172.21.15.23:45856/0 (socket says 172.21.15.23:45856) 2023-03-27T10:02:17.010 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.009+0000 7fed8b7fe700 1 -- 172.21.15.23:0/4256809814 learned_addr learned my addr 172.21.15.23:0/4256809814 (peer_addr_for_me v1:172.21.15.23:0/0) 2023-03-27T10:02:17.011 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.010+0000 7fed90b9c700 1 --2- 172.21.15.23:0/4256809814 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed8c070bb0 0x7fed8c0c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:17.011 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.010+0000 7fed8bfff700 1 --2- 172.21.15.23:0/4256809814 >> v2:172.21.15.112:3300/0 conn(0x7fed8c0c5c90 0x7fed8c0c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:17.011 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.010+0000 7fed8affd700 1 -- 172.21.15.23:0/4256809814 <== mon.0 v1:172.21.15.23:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3025008328 0 0) 0x7fed8c1592a0 con 0x7fed8c159730 2023-03-27T10:02:17.011 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.010+0000 7fed8affd700 1 -- 172.21.15.23:0/4256809814 --> v1:172.21.15.23:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed740035b0 con 0x7fed8c159730 2023-03-27T10:02:17.012 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.010+0000 7fed8bfff700 1 -- 172.21.15.23:0/4256809814 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed8c070bb0 msgr2=0x7fed8c0c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:02:17.012 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.010+0000 7fed8bfff700 1 --2- 172.21.15.23:0/4256809814 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed8c070bb0 0x7fed8c0c6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:17.012 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.010+0000 7fed8bfff700 1 -- 172.21.15.23:0/4256809814 >> v1:172.21.15.23:6789/0 conn(0x7fed8c159730 legacy=0x7fed8c06a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:17.012 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.010+0000 7fed8bfff700 1 -- 172.21.15.23:0/4256809814 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed8c0725b0 con 0x7fed8c0c5c90 2023-03-27T10:02:17.012 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.010+0000 7fed90b9c700 1 --2- 172.21.15.23:0/4256809814 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed8c070bb0 0x7fed8c0c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-27T10:02:17.012 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.010+0000 7fed8bfff700 1 --2- 172.21.15.23:0/4256809814 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed8c0c5c90 0x7fed8c0c8580 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fed7c009840 tx=0x7fed7c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f12fcc012d3944d2 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:17.012 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.011+0000 7fed8affd700 1 -- 172.21.15.23:0/4256809814 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c011830 con 0x7fed8c0c5c90 2023-03-27T10:02:17.013 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.011+0000 7fed8affd700 1 -- 172.21.15.23:0/4256809814 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed7c011990 con 0x7fed8c0c5c90 2023-03-27T10:02:17.013 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.011+0000 7fed8affd700 1 -- 172.21.15.23:0/4256809814 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c011d20 con 0x7fed8c0c5c90 2023-03-27T10:02:17.013 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.011+0000 7fed925ff700 1 -- 172.21.15.23:0/4256809814 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed8c0c5c90 msgr2=0x7fed8c0c8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:17.013 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.011+0000 7fed925ff700 1 --2- 172.21.15.23:0/4256809814 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed8c0c5c90 0x7fed8c0c8580 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fed7c009840 tx=0x7fed7c009b50 comp rx=0 tx=0).stop 2023-03-27T10:02:17.013 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.011+0000 7fed925ff700 1 -- 172.21.15.23:0/4256809814 shutdown_connections 2023-03-27T10:02:17.013 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.011+0000 7fed925ff700 1 --2- 172.21.15.23:0/4256809814 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed8c070bb0 0x7fed8c0c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:17.013 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.011+0000 7fed925ff700 1 --2- 172.21.15.23:0/4256809814 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed8c0c5c90 0x7fed8c0c8580 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:17.014 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.011+0000 7fed925ff700 1 -- 172.21.15.23:0/4256809814 >> 172.21.15.23:0/4256809814 conn(0x7fed8c0cbd30 msgr2=0x7fed8c0c7b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:17.014 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.011+0000 7fed925ff700 1 -- 172.21.15.23:0/4256809814 shutdown_connections 2023-03-27T10:02:17.014 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.011+0000 7fed925ff700 1 -- 172.21.15.23:0/4256809814 wait complete. 2023-03-27T10:02:17.015 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.013+0000 7fed925ff700 1 Processor -- start 2023-03-27T10:02:17.016 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.013+0000 7fed925ff700 1 -- start start 2023-03-27T10:02:17.016 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.014+0000 7fed925ff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fed8c070bb0 0x7fed8c1ef400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:17.016 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.014+0000 7fed925ff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed8c0c5c90 0x7fed8c0c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:17.016 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.014+0000 7fed925ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed8c159730 0x7fed8c1f1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:17.016 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.014+0000 7fed925ff700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8c1519a0 con 0x7fed8c070bb0 2023-03-27T10:02:17.016 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.014+0000 7fed925ff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8c15c740 con 0x7fed8c0c5c90 2023-03-27T10:02:17.017 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.014+0000 7fed925ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed8c1e58e0 con 0x7fed8c159730 2023-03-27T10:02:17.017 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.014+0000 7fed8bfff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fed8c070bb0 0x7fed8c1ef400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:17.017 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.014+0000 7fed8bfff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fed8c070bb0 0x7fed8c1ef400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:48810/0 (socket says 172.21.15.23:48810) 2023-03-27T10:02:17.017 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.014+0000 7fed8bfff700 1 -- 172.21.15.23:0/4264058321 learned_addr learned my addr 172.21.15.23:0/4264058321 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:17.017 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.015+0000 7fed8b7fe700 1 --2- 172.21.15.23:0/4264058321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed8c0c5c90 0x7fed8c0c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:17.017 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.015+0000 7fed90b9c700 1 --2- 172.21.15.23:0/4264058321 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed8c159730 0x7fed8c1f1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:17.017 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.015+0000 7fed8bfff700 1 -- 172.21.15.23:0/4264058321 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed8c159730 msgr2=0x7fed8c1f1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:17.017 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.015+0000 7fed8bfff700 1 --2- 172.21.15.23:0/4264058321 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed8c159730 0x7fed8c1f1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:17.018 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.015+0000 7fed8bfff700 1 -- 172.21.15.23:0/4264058321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed8c0c5c90 msgr2=0x7fed8c0c9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:17.018 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.015+0000 7fed8bfff700 1 --2- 172.21.15.23:0/4264058321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed8c0c5c90 0x7fed8c0c9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:17.018 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.015+0000 7fed8bfff700 1 -- 172.21.15.23:0/4264058321 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed84009a70 con 0x7fed8c070bb0 2023-03-27T10:02:17.018 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.015+0000 7fed8b7fe700 1 --2- 172.21.15.23:0/4264058321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed8c0c5c90 0x7fed8c0c9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-27T10:02:17.018 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.015+0000 7fed8bfff700 1 --2- 172.21.15.23:0/4264058321 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fed8c070bb0 0x7fed8c1ef400 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0090b0 tx=0x7fed7c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:17.018 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.015+0000 7fed88ff9700 1 -- 172.21.15.23:0/4264058321 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c0036a0 con 0x7fed8c070bb0 2023-03-27T10:02:17.018 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.015+0000 7fed88ff9700 1 -- 172.21.15.23:0/4264058321 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed7c01f070 con 0x7fed8c070bb0 2023-03-27T10:02:17.019 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.015+0000 7fed88ff9700 1 -- 172.21.15.23:0/4264058321 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c003a40 con 0x7fed8c070bb0 2023-03-27T10:02:17.019 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.016+0000 7fed925ff700 1 -- 172.21.15.23:0/4264058321 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed7c0094a0 con 0x7fed8c070bb0 2023-03-27T10:02:17.019 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.016+0000 7fed925ff700 1 -- 172.21.15.23:0/4264058321 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed8c158f30 con 0x7fed8c070bb0 2023-03-27T10:02:17.020 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.017+0000 7fed925ff700 1 -- 172.21.15.23:0/4264058321 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed8c158c50 con 0x7fed8c070bb0 2023-03-27T10:02:17.020 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.017+0000 7fed88ff9700 1 -- 172.21.15.23:0/4264058321 <== mon.0 v2:172.21.15.23:3300/0 4 ==== mgrmap(e 6) v1 ==== 85113+0+0 (secure 0 0 0) 0x7fed7c00f050 con 0x7fed8c070bb0 2023-03-27T10:02:17.020 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.018+0000 7fed88ff9700 1 --2- 172.21.15.23:0/4264058321 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fed78066e60 0x7fed78069310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:17.020 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.018+0000 7fed88ff9700 1 -- 172.21.15.23:0/4264058321 <== mon.0 v2:172.21.15.23:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6781+0+0 (secure 0 0 0) 0x7fed7c021070 con 0x7fed8c070bb0 2023-03-27T10:02:17.020 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.019+0000 7fed8b7fe700 1 --2- 172.21.15.23:0/4264058321 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fed78066e60 0x7fed78069310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:17.020 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.019+0000 7fed8b7fe700 1 --2- 172.21.15.23:0/4264058321 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fed78066e60 0x7fed78069310 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fed8c151f80 tx=0x7fed80009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:17.022 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.023+0000 7fed88ff9700 1 -- 172.21.15.23:0/4264058321 <== mon.0 v2:172.21.15.23:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173280 (secure 0 0 0) 0x7fed7c052a50 con 0x7fed8c070bb0 2023-03-27T10:02:17.272 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.273+0000 7fed925ff700 1 -- 172.21.15.23:0/4264058321 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7fed8c0c65a0 con 0x7fed8c070bb0 2023-03-27T10:02:17.277 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.277+0000 7fed88ff9700 1 -- 172.21.15.23:0/4264058321 <== mon.0 v2:172.21.15.23:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 5 to fsmap v5) v1 ==== 137+0+0 (secure 0 0 0) 0x7fed7c015020 con 0x7fed8c070bb0 2023-03-27T10:02:17.280 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.280+0000 7fed925ff700 1 -- 172.21.15.23:0/4264058321 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fed78066e60 msgr2=0x7fed78069310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:17.280 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.280+0000 7fed925ff700 1 --2- 172.21.15.23:0/4264058321 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fed78066e60 0x7fed78069310 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fed8c151f80 tx=0x7fed80009040 comp rx=0 tx=0).stop 2023-03-27T10:02:17.280 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.280+0000 7fed925ff700 1 -- 172.21.15.23:0/4264058321 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fed8c070bb0 msgr2=0x7fed8c1ef400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:17.281 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.280+0000 7fed925ff700 1 --2- 172.21.15.23:0/4264058321 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fed8c070bb0 0x7fed8c1ef400 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0090b0 tx=0x7fed7c004750 comp rx=0 tx=0).stop 2023-03-27T10:02:17.281 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.280+0000 7fed925ff700 1 -- 172.21.15.23:0/4264058321 shutdown_connections 2023-03-27T10:02:17.281 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.280+0000 7fed925ff700 1 --2- 172.21.15.23:0/4264058321 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fed78066e60 0x7fed78069310 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:17.281 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.280+0000 7fed925ff700 1 --2- 172.21.15.23:0/4264058321 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed8c159730 0x7fed8c1f1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:17.281 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.280+0000 7fed925ff700 1 --2- 172.21.15.23:0/4264058321 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fed8c070bb0 0x7fed8c1ef400 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:17.281 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.280+0000 7fed925ff700 1 --2- 172.21.15.23:0/4264058321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed8c0c5c90 0x7fed8c0c9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:17.282 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.280+0000 7fed925ff700 1 -- 172.21.15.23:0/4264058321 >> 172.21.15.23:0/4264058321 conn(0x7fed8c0cbd30 msgr2=0x7fed8c06a240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:17.282 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.281+0000 7fed925ff700 1 -- 172.21.15.23:0/4264058321 shutdown_connections 2023-03-27T10:02:17.282 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.281+0000 7fed925ff700 1 -- 172.21.15.23:0/4264058321 wait complete. 2023-03-27T10:02:17.282 INFO:teuthology.orchestra.run.smithi023.stderr:added data pool 5 to fsmap 2023-03-27T10:02:17.297 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-03-27T10:02:17.492 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.493+0000 7f87b5bff700 1 Processor -- start 2023-03-27T10:02:17.493 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.493+0000 7f87b5bff700 1 -- start start 2023-03-27T10:02:17.494 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.494+0000 7f87b5bff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87b00c6990 0x7f87b00c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:17.494 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.494+0000 7f87b5bff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f87b00c9520 0x7f87b00c8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:17.494 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.494+0000 7f87b5bff700 1 -- --> v1:172.21.15.23:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b00ca730 con 0x7f87b006b440 2023-03-27T10:02:17.494 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.494+0000 7f87b5bff700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87b00c72a0 con 0x7f87b00c9520 2023-03-27T10:02:17.494 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.494+0000 7f87b5bff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b00cf9f0 con 0x7f87b00c6990 2023-03-27T10:02:17.495 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.494+0000 7f87affff700 1 --1- >> v1:172.21.15.23:6789/0 conn(0x7f87b006b440 0x7f87b006b860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.23:6789/0 says I am v1:172.21.15.23:36710/0 (socket says 172.21.15.23:36710) 2023-03-27T10:02:17.495 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.494+0000 7f87affff700 1 -- 172.21.15.23:0/3015491973 learned_addr learned my addr 172.21.15.23:0/3015491973 (peer_addr_for_me v1:172.21.15.23:0/0) 2023-03-27T10:02:17.495 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.494+0000 7f87af7fe700 1 --2- 172.21.15.23:0/3015491973 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87b00c6990 0x7f87b00c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:17.495 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.494+0000 7f87aeffd700 1 --2- 172.21.15.23:0/3015491973 >> v2:172.21.15.112:3300/0 conn(0x7f87b00c9520 0x7f87b00c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:17.496 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.495+0000 7f87ae7fc700 1 -- 172.21.15.23:0/3015491973 <== mon.0 v1:172.21.15.23:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3758666481 0 0) 0x7f87b00ca730 con 0x7f87b006b440 2023-03-27T10:02:17.496 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.495+0000 7f87ae7fc700 1 -- 172.21.15.23:0/3015491973 --> v1:172.21.15.23:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87980035b0 con 0x7f87b006b440 2023-03-27T10:02:17.497 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.495+0000 7f87aeffd700 1 -- 172.21.15.23:0/3015491973 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87b00c6990 msgr2=0x7f87b00c6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:02:17.497 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.495+0000 7f87aeffd700 1 --2- 172.21.15.23:0/3015491973 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87b00c6990 0x7f87b00c6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:17.497 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.495+0000 7f87aeffd700 1 -- 172.21.15.23:0/3015491973 >> v1:172.21.15.23:6789/0 conn(0x7f87b006b440 legacy=0x7f87b006b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:17.497 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.495+0000 7f87aeffd700 1 -- 172.21.15.23:0/3015491973 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b0158a00 con 0x7f87b00c9520 2023-03-27T10:02:17.497 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.495+0000 7f87af7fe700 1 --2- 172.21.15.23:0/3015491973 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87b00c6990 0x7f87b00c6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-27T10:02:17.497 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.495+0000 7f87aeffd700 1 --2- 172.21.15.23:0/3015491973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b00c9520 0x7f87b00c8820 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f87a4009840 tx=0x7f87a4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66b6e8acc9f138f7 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:17.498 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.495+0000 7f87ae7fc700 1 -- 172.21.15.23:0/3015491973 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a4011830 con 0x7f87b00c9520 2023-03-27T10:02:17.498 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.495+0000 7f87ae7fc700 1 -- 172.21.15.23:0/3015491973 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a4011990 con 0x7f87b00c9520 2023-03-27T10:02:17.498 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.495+0000 7f87ae7fc700 1 -- 172.21.15.23:0/3015491973 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a4011d00 con 0x7f87b00c9520 2023-03-27T10:02:17.498 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.496+0000 7f87b5bff700 1 -- 172.21.15.23:0/3015491973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b00c9520 msgr2=0x7f87b00c8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:17.498 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.496+0000 7f87b5bff700 1 --2- 172.21.15.23:0/3015491973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b00c9520 0x7f87b00c8820 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f87a4009840 tx=0x7f87a4009b50 comp rx=0 tx=0).stop 2023-03-27T10:02:17.498 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.496+0000 7f87b5bff700 1 -- 172.21.15.23:0/3015491973 shutdown_connections 2023-03-27T10:02:17.498 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.496+0000 7f87b5bff700 1 --2- 172.21.15.23:0/3015491973 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87b00c6990 0x7f87b00c6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:17.498 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.496+0000 7f87b5bff700 1 --2- 172.21.15.23:0/3015491973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b00c9520 0x7f87b00c8820 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:17.499 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.496+0000 7f87b5bff700 1 -- 172.21.15.23:0/3015491973 >> 172.21.15.23:0/3015491973 conn(0x7f87b0071920 msgr2=0x7f87b0071cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:17.499 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.496+0000 7f87b5bff700 1 -- 172.21.15.23:0/3015491973 shutdown_connections 2023-03-27T10:02:17.499 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.496+0000 7f87b5bff700 1 -- 172.21.15.23:0/3015491973 wait complete. 2023-03-27T10:02:17.500 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.498+0000 7f87b5bff700 1 Processor -- start 2023-03-27T10:02:17.500 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.498+0000 7f87b5bff700 1 -- start start 2023-03-27T10:02:17.500 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.499+0000 7f87b5bff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87b006b440 0x7f87b014b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:17.500 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.499+0000 7f87b5bff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f87b00c6990 0x7f87b014c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:17.500 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.499+0000 7f87b5bff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b00c9520 0x7f87b014c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:17.500 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.499+0000 7f87b5bff700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b0153440 con 0x7f87b00c6990 2023-03-27T10:02:17.500 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.499+0000 7f87b5bff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b014b1c0 con 0x7f87b00c9520 2023-03-27T10:02:17.501 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.499+0000 7f87b5bff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b014cea0 con 0x7f87b006b440 2023-03-27T10:02:17.502 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.499+0000 7f87aeffd700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f87b00c6990 0x7f87b014c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:17.502 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.499+0000 7f87aeffd700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f87b00c6990 0x7f87b014c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:34518/0 (socket says 172.21.15.23:34518) 2023-03-27T10:02:17.503 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.499+0000 7f87af7fe700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87b006b440 0x7f87b014b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:17.503 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.499+0000 7f87aeffd700 1 -- 172.21.15.23:0/612850744 learned_addr learned my addr 172.21.15.23:0/612850744 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:17.503 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.499+0000 7f87affff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b00c9520 0x7f87b014c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:17.503 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.500+0000 7f87aeffd700 1 -- 172.21.15.23:0/612850744 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87b006b440 msgr2=0x7f87b014b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:17.503 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.500+0000 7f87aeffd700 1 --2- 172.21.15.23:0/612850744 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87b006b440 0x7f87b014b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:17.503 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.500+0000 7f87aeffd700 1 -- 172.21.15.23:0/612850744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b00c9520 msgr2=0x7f87b014c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:17.503 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.500+0000 7f87aeffd700 1 --2- 172.21.15.23:0/612850744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b00c9520 0x7f87b014c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:17.504 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.500+0000 7f87aeffd700 1 -- 172.21.15.23:0/612850744 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a00094a0 con 0x7f87b00c6990 2023-03-27T10:02:17.504 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.500+0000 7f87affff700 1 --2- 172.21.15.23:0/612850744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b00c9520 0x7f87b014c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-27T10:02:17.504 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.500+0000 7f87aeffd700 1 --2- 172.21.15.23:0/612850744 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f87b00c6990 0x7f87b014c420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f87a40090b0 tx=0x7f87a4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:17.504 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.500+0000 7f8797fff700 1 -- 172.21.15.23:0/612850744 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a40044d0 con 0x7f87b00c6990 2023-03-27T10:02:17.504 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.500+0000 7f8797fff700 1 -- 172.21.15.23:0/612850744 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a401f070 con 0x7f87b00c6990 2023-03-27T10:02:17.504 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.500+0000 7f8797fff700 1 -- 172.21.15.23:0/612850744 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a40038a0 con 0x7f87b00c6990 2023-03-27T10:02:17.505 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.500+0000 7f87b5bff700 1 -- 172.21.15.23:0/612850744 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87a40094a0 con 0x7f87b00c6990 2023-03-27T10:02:17.505 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.500+0000 7f87b5bff700 1 -- 172.21.15.23:0/612850744 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b0158fc0 con 0x7f87b00c6990 2023-03-27T10:02:17.505 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.501+0000 7f87b5bff700 1 -- 172.21.15.23:0/612850744 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b0158ce0 con 0x7f87b00c6990 2023-03-27T10:02:17.505 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.502+0000 7f8797fff700 1 -- 172.21.15.23:0/612850744 <== mon.0 v2:172.21.15.23:3300/0 4 ==== mgrmap(e 6) v1 ==== 85113+0+0 (secure 0 0 0) 0x7f87a400f050 con 0x7f87b00c6990 2023-03-27T10:02:17.505 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.503+0000 7f8797fff700 1 --2- 172.21.15.23:0/612850744 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f879c066e30 0x7f879c0692e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:17.505 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.503+0000 7f8797fff700 1 -- 172.21.15.23:0/612850744 <== mon.0 v2:172.21.15.23:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f87a4021070 con 0x7f87b00c6990 2023-03-27T10:02:17.505 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.503+0000 7f87af7fe700 1 --2- 172.21.15.23:0/612850744 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f879c066e30 0x7f879c0692e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:17.506 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.504+0000 7f87af7fe700 1 --2- 172.21.15.23:0/612850744 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f879c066e30 0x7f879c0692e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f87a0009d70 tx=0x7f87a0009210 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:17.506 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.507+0000 7f8797fff700 1 -- 172.21.15.23:0/612850744 <== mon.0 v2:172.21.15.23:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173280 (secure 0 0 0) 0x7f87a4052b90 con 0x7f87b00c6990 2023-03-27T10:02:17.750 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.750+0000 7f87b5bff700 1 -- 172.21.15.23:0/612850744 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87b01592a0 con 0x7f87b00c6990 2023-03-27T10:02:17.750 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.751+0000 7f8797fff700 1 -- 172.21.15.23:0/612850744 <== mon.0 v2:172.21.15.23:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+6128 (secure 0 0 0) 0x7f87a402b090 con 0x7f87b00c6990 2023-03-27T10:02:17.751 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:02:17.751 INFO:teuthology.orchestra.run.smithi023.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.23:6836/4002126418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6834","nonce":4002126418},{"type":"v1","addr":"172.21.15.23:6836","nonce":4002126418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4253,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.23:6837/1693839903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6835","nonce":1693839903},{"type":"v1","addr":"172.21.15.23:6837","nonce":1693839903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4294,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/357813579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":357813579},{"type":"v1","addr":"172.21.15.112:6836","nonce":357813579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4297,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/394077155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":394077155},{"type":"v1","addr":"172.21.15.112:6837","nonce":394077155}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4300,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/538239518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":538239518},{"type":"v1","addr":"172.21.15.112:6839","nonce":538239518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-03-27T10:02:12.325598+0000","modified":"2023-03-27T10:02:17.274862+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"a","rank":0,"incarnation":4,"state":"up:creating","state_seq":1,"addr":"172.21.15.23:6833/1046971212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6832","nonce":1046971212},{"type":"v1","addr":"172.21.15.23:6833","nonce":1046971212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-03-27T10:02:17.753 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.753+0000 7f87b5bff700 1 -- 172.21.15.23:0/612850744 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f879c066e30 msgr2=0x7f879c0692e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:17.753 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.753+0000 7f87b5bff700 1 --2- 172.21.15.23:0/612850744 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f879c066e30 0x7f879c0692e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f87a0009d70 tx=0x7f87a0009210 comp rx=0 tx=0).stop 2023-03-27T10:02:17.753 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.753+0000 7f87b5bff700 1 -- 172.21.15.23:0/612850744 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f87b00c6990 msgr2=0x7f87b014c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:17.753 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.753+0000 7f87b5bff700 1 --2- 172.21.15.23:0/612850744 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f87b00c6990 0x7f87b014c420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f87a40090b0 tx=0x7f87a4004750 comp rx=0 tx=0).stop 2023-03-27T10:02:17.753 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.753+0000 7f87b5bff700 1 -- 172.21.15.23:0/612850744 shutdown_connections 2023-03-27T10:02:17.753 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.753+0000 7f87b5bff700 1 --2- 172.21.15.23:0/612850744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b00c9520 0x7f87b014c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:17.754 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.753+0000 7f87b5bff700 1 --2- 172.21.15.23:0/612850744 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f879c066e30 0x7f879c0692e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:17.754 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.753+0000 7f87b5bff700 1 --2- 172.21.15.23:0/612850744 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87b006b440 0x7f87b014b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:17.754 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.753+0000 7f87b5bff700 1 --2- 172.21.15.23:0/612850744 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f87b00c6990 0x7f87b014c420 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:17.754 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.753+0000 7f87b5bff700 1 -- 172.21.15.23:0/612850744 >> 172.21.15.23:0/612850744 conn(0x7f87b0071920 msgr2=0x7f87b00cabd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:17.754 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.753+0000 7f87b5bff700 1 -- 172.21.15.23:0/612850744 shutdown_connections 2023-03-27T10:02:17.754 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.753+0000 7f87b5bff700 1 -- 172.21.15.23:0/612850744 wait complete. 2023-03-27T10:02:17.755 INFO:teuthology.orchestra.run.smithi023.stderr:dumped fsmap epoch 5 2023-03-27T10:02:17.770 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-27T10:02:17.959 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.960+0000 7fa54e5ff700 1 Processor -- start 2023-03-27T10:02:17.959 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.960+0000 7fa54e5ff700 1 -- start start 2023-03-27T10:02:17.960 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.961+0000 7fa54e5ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa5480c6990 0x7fa5480c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:17.960 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.961+0000 7fa54e5ff700 1 -- --> v1:172.21.15.23:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5480ca730 con 0x7fa54806b440 2023-03-27T10:02:17.960 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.961+0000 7fa54e5ff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5480c8f30 con 0x7fa5480c9520 2023-03-27T10:02:17.961 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.961+0000 7fa54e5ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5480c72a0 con 0x7fa5480c6990 2023-03-27T10:02:17.962 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.961+0000 7fa54cb9c700 1 --1- >> v1:172.21.15.23:6789/0 conn(0x7fa54806b440 0x7fa54806b860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.23:6789/0 says I am v1:172.21.15.23:36718/0 (socket says 172.21.15.23:36718) 2023-03-27T10:02:17.962 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.961+0000 7fa54cb9c700 1 -- 172.21.15.23:0/329648620 learned_addr learned my addr 172.21.15.23:0/329648620 (peer_addr_for_me v1:172.21.15.23:0/0) 2023-03-27T10:02:17.962 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.961+0000 7fa547fff700 1 --2- 172.21.15.23:0/329648620 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa5480c6990 0x7fa5480c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:17.962 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.962+0000 7fa546ffd700 1 -- 172.21.15.23:0/329648620 <== mon.0 v1:172.21.15.23:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1162919903 0 0) 0x7fa5480ca730 con 0x7fa54806b440 2023-03-27T10:02:17.962 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.962+0000 7fa546ffd700 1 -- 172.21.15.23:0/329648620 --> v1:172.21.15.23:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5300035b0 con 0x7fa54806b440 2023-03-27T10:02:17.963 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.962+0000 7fa546ffd700 1 -- 172.21.15.23:0/329648620 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3877085401 0 0) 0x7fa5480c8f30 con 0x7fa5480c9520 2023-03-27T10:02:17.963 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.962+0000 7fa546ffd700 1 -- 172.21.15.23:0/329648620 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5480ca730 con 0x7fa5480c9520 2023-03-27T10:02:17.963 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.962+0000 7fa547fff700 1 -- 172.21.15.23:0/329648620 >> v1:172.21.15.112:6789/0 conn(0x7fa5480c9520 legacy=0x7fa5480c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:17.963 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.962+0000 7fa547fff700 1 -- 172.21.15.23:0/329648620 >> v1:172.21.15.23:6789/0 conn(0x7fa54806b440 legacy=0x7fa54806b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:17.963 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.962+0000 7fa547fff700 1 -- 172.21.15.23:0/329648620 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5480cf9f0 con 0x7fa5480c6990 2023-03-27T10:02:17.963 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.962+0000 7fa547fff700 1 --2- 172.21.15.23:0/329648620 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa5480c6990 0x7fa5480c6d60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa538009870 tx=0x7fa538009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=817817d460eaafa0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:17.963 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.962+0000 7fa546ffd700 1 -- 172.21.15.23:0/329648620 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa538011830 con 0x7fa5480c6990 2023-03-27T10:02:17.963 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.962+0000 7fa546ffd700 1 -- 172.21.15.23:0/329648620 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa538011990 con 0x7fa5480c6990 2023-03-27T10:02:17.964 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.962+0000 7fa546ffd700 1 -- 172.21.15.23:0/329648620 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa538011d00 con 0x7fa5480c6990 2023-03-27T10:02:17.964 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.963+0000 7fa54e5ff700 1 -- 172.21.15.23:0/329648620 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa5480c6990 msgr2=0x7fa5480c6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:17.965 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.963+0000 7fa54e5ff700 1 --2- 172.21.15.23:0/329648620 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa5480c6990 0x7fa5480c6d60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa538009870 tx=0x7fa538009b80 comp rx=0 tx=0).stop 2023-03-27T10:02:17.965 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.963+0000 7fa54e5ff700 1 -- 172.21.15.23:0/329648620 shutdown_connections 2023-03-27T10:02:17.965 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.963+0000 7fa54e5ff700 1 --2- 172.21.15.23:0/329648620 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa5480c6990 0x7fa5480c6d60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:17.965 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.963+0000 7fa54e5ff700 1 -- 172.21.15.23:0/329648620 >> 172.21.15.23:0/329648620 conn(0x7fa548071920 msgr2=0x7fa548071cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:17.965 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.963+0000 7fa54e5ff700 1 -- 172.21.15.23:0/329648620 shutdown_connections 2023-03-27T10:02:17.966 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.963+0000 7fa54e5ff700 1 -- 172.21.15.23:0/329648620 wait complete. 2023-03-27T10:02:17.966 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.965+0000 7fa54e5ff700 1 Processor -- start 2023-03-27T10:02:17.966 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.965+0000 7fa54e5ff700 1 -- start start 2023-03-27T10:02:17.967 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.966+0000 7fa54e5ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa54806b440 0x7fa5481ef4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:17.968 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.966+0000 7fa54e5ff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5480c6990 0x7fa5481efa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:17.968 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.966+0000 7fa54e5ff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fa5480c9520 0x7fa54814d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:17.968 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.966+0000 7fa54e5ff700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54815c720 con 0x7fa5480c9520 2023-03-27T10:02:17.968 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.966+0000 7fa54e5ff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54815bb50 con 0x7fa5480c6990 2023-03-27T10:02:17.968 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.966+0000 7fa54e5ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa548158130 con 0x7fa54806b440 2023-03-27T10:02:17.968 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.966+0000 7fa54cb9c700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fa5480c9520 0x7fa54814d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:17.968 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.966+0000 7fa547fff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa54806b440 0x7fa5481ef4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:17.969 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.966+0000 7fa5477fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5480c6990 0x7fa5481efa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:17.969 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.966+0000 7fa547fff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa54806b440 0x7fa5481ef4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.23:43732/0 (socket says 172.21.15.23:43732) 2023-03-27T10:02:17.969 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.966+0000 7fa547fff700 1 -- 172.21.15.23:0/2561123663 learned_addr learned my addr 172.21.15.23:0/2561123663 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:17.969 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.966+0000 7fa54cb9c700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fa5480c9520 0x7fa54814d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:34526/0 (socket says 172.21.15.23:34526) 2023-03-27T10:02:17.969 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.966+0000 7fa547fff700 1 -- 172.21.15.23:0/2561123663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5480c6990 msgr2=0x7fa5481efa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:17.969 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.966+0000 7fa547fff700 1 --2- 172.21.15.23:0/2561123663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5480c6990 0x7fa5481efa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:17.969 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.966+0000 7fa547fff700 1 -- 172.21.15.23:0/2561123663 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fa5480c9520 msgr2=0x7fa54814d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:17.970 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.966+0000 7fa547fff700 1 --2- 172.21.15.23:0/2561123663 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fa5480c9520 0x7fa54814d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:17.970 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.966+0000 7fa547fff700 1 -- 172.21.15.23:0/2561123663 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5380094a0 con 0x7fa54806b440 2023-03-27T10:02:17.970 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.967+0000 7fa547fff700 1 --2- 172.21.15.23:0/2561123663 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa54806b440 0x7fa5481ef4f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa548072540 tx=0x7fa538004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:17.970 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.967+0000 7fa544ff9700 1 -- 172.21.15.23:0/2561123663 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5380044d0 con 0x7fa54806b440 2023-03-27T10:02:17.970 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.967+0000 7fa544ff9700 1 -- 172.21.15.23:0/2561123663 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa53801f070 con 0x7fa54806b440 2023-03-27T10:02:17.970 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.967+0000 7fa544ff9700 1 -- 172.21.15.23:0/2561123663 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5380038c0 con 0x7fa54806b440 2023-03-27T10:02:17.970 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.967+0000 7fa54e5ff700 1 -- 172.21.15.23:0/2561123663 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa548158ce0 con 0x7fa54806b440 2023-03-27T10:02:17.970 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.967+0000 7fa54e5ff700 1 -- 172.21.15.23:0/2561123663 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa548158a00 con 0x7fa54806b440 2023-03-27T10:02:17.971 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.968+0000 7fa54e5ff700 1 -- 172.21.15.23:0/2561123663 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa548158fc0 con 0x7fa54806b440 2023-03-27T10:02:17.971 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.969+0000 7fa544ff9700 1 -- 172.21.15.23:0/2561123663 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 6) v1 ==== 85113+0+0 (secure 0 0 0) 0x7fa53800f050 con 0x7fa54806b440 2023-03-27T10:02:17.971 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.970+0000 7fa544ff9700 1 --2- 172.21.15.23:0/2561123663 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fa534066e30 0x7fa5340692e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:17.972 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.970+0000 7fa544ff9700 1 -- 172.21.15.23:0/2561123663 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa538021070 con 0x7fa54806b440 2023-03-27T10:02:17.972 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.970+0000 7fa5477fe700 1 --2- 172.21.15.23:0/2561123663 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fa534066e30 0x7fa5340692e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:17.972 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.971+0000 7fa5477fe700 1 --2- 172.21.15.23:0/2561123663 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fa534066e30 0x7fa5340692e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa5480cc4e0 tx=0x7fa53c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:17.975 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:17.975+0000 7fa544ff9700 1 -- 172.21.15.23:0/2561123663 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173280 (secure 0 0 0) 0x7fa538052c40 con 0x7fa54806b440 2023-03-27T10:02:18.176 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.177+0000 7fa54e5ff700 1 -- 172.21.15.23:0/2561123663 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5481e4a70 con 0x7fa54806b440 2023-03-27T10:02:18.179 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.178+0000 7fa544ff9700 1 -- 172.21.15.23:0/2561123663 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+19798 (secure 0 0 0) 0x7fa538015090 con 0x7fa54806b440 2023-03-27T10:02:18.180 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:02:18.180 INFO:teuthology.orchestra.run.smithi023.stdout:{"epoch":24,"fsid":"1d8d6e6e-0d21-4f51-91ee-d9434d4fb750","created":"2023-03-27T10:01:53.805778+0000","modified":"2023-03-27T10:02:17.274631+0000","last_up_change":"2023-03-27T10:02:04.172266+0000","last_in_change":"2023-03-27T10:01:56.475091+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-27T10:02:05.288883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-27T10:02:05.643442+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-27T10:02:10.720339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-27T10:02:11.254847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-27T10:02:14.809537+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"df086052-536b-4fb1-bcce-7aaf5a13911e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6800","nonce":96642},{"type":"v1","addr":"172.21.15.23:6801","nonce":96642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6802","nonce":96642},{"type":"v1","addr":"172.21.15.23:6803","nonce":96642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6806","nonce":96642},{"type":"v1","addr":"172.21.15.23:6807","nonce":96642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6804","nonce":96642},{"type":"v1","addr":"172.21.15.23:6805","nonce":96642}]},"public_addr":"172.21.15.23:6801/96642","cluster_addr":"172.21.15.23:6803/96642","heartbeat_back_addr":"172.21.15.23:6807/96642","heartbeat_front_addr":"172.21.15.23:6805/96642","state":["exists","up"]},{"osd":1,"uuid":"03138001-a3fd-43fc-9797-a47643d9a3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6808","nonce":96643},{"type":"v1","addr":"172.21.15.23:6809","nonce":96643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6810","nonce":96643},{"type":"v1","addr":"172.21.15.23:6811","nonce":96643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6814","nonce":96643},{"type":"v1","addr":"172.21.15.23:6815","nonce":96643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6812","nonce":96643},{"type":"v1","addr":"172.21.15.23:6813","nonce":96643}]},"public_addr":"172.21.15.23:6809/96643","cluster_addr":"172.21.15.23:6811/96643","heartbeat_back_addr":"172.21.15.23:6815/96643","heartbeat_front_addr":"172.21.15.23:6813/96643","state":["exists","up"]},{"osd":2,"uuid":"964dd926-1160-44ef-aa34-b18965dc05f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6817","nonce":96644},{"type":"v1","addr":"172.21.15.23:6819","nonce":96644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6820","nonce":96644},{"type":"v1","addr":"172.21.15.23:6822","nonce":96644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6827","nonce":96644},{"type":"v1","addr":"172.21.15.23:6829","nonce":96644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6824","nonce":96644},{"type":"v1","addr":"172.21.15.23:6825","nonce":96644}]},"public_addr":"172.21.15.23:6819/96644","cluster_addr":"172.21.15.23:6822/96644","heartbeat_back_addr":"172.21.15.23:6829/96644","heartbeat_front_addr":"172.21.15.23:6825/96644","state":["exists","up"]},{"osd":3,"uuid":"4ae5500f-cb24-44a8-a536-5c399c09e566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6816","nonce":96645},{"type":"v1","addr":"172.21.15.23:6818","nonce":96645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6821","nonce":96645},{"type":"v1","addr":"172.21.15.23:6823","nonce":96645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6830","nonce":96645},{"type":"v1","addr":"172.21.15.23:6831","nonce":96645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6826","nonce":96645},{"type":"v1","addr":"172.21.15.23:6828","nonce":96645}]},"public_addr":"172.21.15.23:6818/96645","cluster_addr":"172.21.15.23:6823/96645","heartbeat_back_addr":"172.21.15.23:6831/96645","heartbeat_front_addr":"172.21.15.23:6828/96645","state":["exists","up"]},{"osd":4,"uuid":"de228ab2-92b4-40bc-8dbd-8f2f4895dc22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":127986},{"type":"v1","addr":"172.21.15.112:6809","nonce":127986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":127986},{"type":"v1","addr":"172.21.15.112:6811","nonce":127986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6815","nonce":127986},{"type":"v1","addr":"172.21.15.112:6818","nonce":127986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":127986},{"type":"v1","addr":"172.21.15.112:6813","nonce":127986}]},"public_addr":"172.21.15.112:6809/127986","cluster_addr":"172.21.15.112:6811/127986","heartbeat_back_addr":"172.21.15.112:6818/127986","heartbeat_front_addr":"172.21.15.112:6813/127986","state":["exists","up"]},{"osd":5,"uuid":"6c839571-b490-4e66-92ac-0d7301c2e677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":127987},{"type":"v1","addr":"172.21.15.112:6801","nonce":127987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":127987},{"type":"v1","addr":"172.21.15.112:6803","nonce":127987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":127987},{"type":"v1","addr":"172.21.15.112:6807","nonce":127987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":127987},{"type":"v1","addr":"172.21.15.112:6805","nonce":127987}]},"public_addr":"172.21.15.112:6801/127987","cluster_addr":"172.21.15.112:6803/127987","heartbeat_back_addr":"172.21.15.112:6807/127987","heartbeat_front_addr":"172.21.15.112:6805/127987","state":["exists","up"]},{"osd":6,"uuid":"871b1a2b-f0ed-4faf-8f9a-a5a0973e11c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":127988},{"type":"v1","addr":"172.21.15.112:6816","nonce":127988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":127988},{"type":"v1","addr":"172.21.15.112:6819","nonce":127988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":127988},{"type":"v1","addr":"172.21.15.112:6823","nonce":127988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":127988},{"type":"v1","addr":"172.21.15.112:6821","nonce":127988}]},"public_addr":"172.21.15.112:6816/127988","cluster_addr":"172.21.15.112:6819/127988","heartbeat_back_addr":"172.21.15.112:6823/127988","heartbeat_front_addr":"172.21.15.112:6821/127988","state":["exists","up"]},{"osd":7,"uuid":"d8228353-34b5-4363-84d8-642512f5bc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":128009},{"type":"v1","addr":"172.21.15.112:6825","nonce":128009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":128009},{"type":"v1","addr":"172.21.15.112:6827","nonce":128009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":128009},{"type":"v1","addr":"172.21.15.112:6831","nonce":128009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":128009},{"type":"v1","addr":"172.21.15.112:6829","nonce":128009}]},"public_addr":"172.21.15.112:6825/128009","cluster_addr":"172.21.15.112:6827/128009","heartbeat_back_addr":"172.21.15.112:6831/128009","heartbeat_front_addr":"172.21.15.112:6829/128009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:02.132160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:01.882517+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:01.927673+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:01.867525+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:01.723580+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:01.730394+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:01.716687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:01.744322+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-27T10:02:18.181 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.181+0000 7fa54e5ff700 1 -- 172.21.15.23:0/2561123663 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fa534066e30 msgr2=0x7fa5340692e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:18.181 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.181+0000 7fa54e5ff700 1 --2- 172.21.15.23:0/2561123663 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fa534066e30 0x7fa5340692e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa5480cc4e0 tx=0x7fa53c009040 comp rx=0 tx=0).stop 2023-03-27T10:02:18.182 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.181+0000 7fa54e5ff700 1 -- 172.21.15.23:0/2561123663 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa54806b440 msgr2=0x7fa5481ef4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:18.182 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.181+0000 7fa54e5ff700 1 --2- 172.21.15.23:0/2561123663 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa54806b440 0x7fa5481ef4f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa548072540 tx=0x7fa538004750 comp rx=0 tx=0).stop 2023-03-27T10:02:18.182 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.181+0000 7fa54e5ff700 1 -- 172.21.15.23:0/2561123663 shutdown_connections 2023-03-27T10:02:18.182 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.181+0000 7fa54e5ff700 1 --2- 172.21.15.23:0/2561123663 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fa5480c9520 0x7fa54814d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:18.182 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.181+0000 7fa54e5ff700 1 --2- 172.21.15.23:0/2561123663 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fa534066e30 0x7fa5340692e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:18.182 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.181+0000 7fa54e5ff700 1 --2- 172.21.15.23:0/2561123663 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa54806b440 0x7fa5481ef4f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:18.183 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.181+0000 7fa54e5ff700 1 --2- 172.21.15.23:0/2561123663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5480c6990 0x7fa5481efa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:18.183 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.181+0000 7fa54e5ff700 1 -- 172.21.15.23:0/2561123663 >> 172.21.15.23:0/2561123663 conn(0x7fa548071920 msgr2=0x7fa5480cb730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:18.183 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.181+0000 7fa54e5ff700 1 -- 172.21.15.23:0/2561123663 shutdown_connections 2023-03-27T10:02:18.183 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.181+0000 7fa54e5ff700 1 -- 172.21.15.23:0/2561123663 wait complete. 2023-03-27T10:02:18.198 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-27T10:02:18.387 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.387+0000 7f63b15ff700 1 Processor -- start 2023-03-27T10:02:18.387 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.387+0000 7f63b15ff700 1 -- start start 2023-03-27T10:02:18.388 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.388+0000 7f63b15ff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f63ac0c9520 0x7f63ac0c8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:18.388 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.388+0000 7f63b15ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63ac06b440 0x7f63ac06b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:18.388 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.388+0000 7f63b15ff700 1 -- --> v1:172.21.15.23:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63ac0ca730 con 0x7f63ac0c6990 2023-03-27T10:02:18.388 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.389+0000 7f63b15ff700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63ac0cf9f0 con 0x7f63ac0c9520 2023-03-27T10:02:18.389 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.389+0000 7f63b15ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac158a00 con 0x7f63ac06b440 2023-03-27T10:02:18.389 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.389+0000 7f63aaffd700 1 --1- >> v1:172.21.15.23:6789/0 conn(0x7f63ac0c6990 0x7f63ac0c6d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.23:6789/0 says I am v1:172.21.15.23:36724/0 (socket says 172.21.15.23:36724) 2023-03-27T10:02:18.389 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.389+0000 7f63aaffd700 1 -- 172.21.15.23:0/3512449740 learned_addr learned my addr 172.21.15.23:0/3512449740 (peer_addr_for_me v1:172.21.15.23:0/0) 2023-03-27T10:02:18.389 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.389+0000 7f63ab7fe700 1 --2- 172.21.15.23:0/3512449740 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63ac06b440 0x7f63ac06b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:18.390 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.389+0000 7f63aa7fc700 1 --2- 172.21.15.23:0/3512449740 >> v2:172.21.15.112:3300/0 conn(0x7f63ac0c9520 0x7f63ac0c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:18.391 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.389+0000 7f63a9ffb700 1 -- 172.21.15.23:0/3512449740 <== mon.0 v1:172.21.15.23:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3000518725 0 0) 0x7f63ac0ca730 con 0x7f63ac0c6990 2023-03-27T10:02:18.391 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.389+0000 7f63a9ffb700 1 -- 172.21.15.23:0/3512449740 --> v1:172.21.15.23:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63940035b0 con 0x7f63ac0c6990 2023-03-27T10:02:18.391 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.389+0000 7f63ab7fe700 1 -- 172.21.15.23:0/3512449740 >> v2:172.21.15.112:3300/0 conn(0x7f63ac0c9520 msgr2=0x7f63ac0c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:02:18.392 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.389+0000 7f63ab7fe700 1 --2- 172.21.15.23:0/3512449740 >> v2:172.21.15.112:3300/0 conn(0x7f63ac0c9520 0x7f63ac0c8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:18.392 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.389+0000 7f63ab7fe700 1 -- 172.21.15.23:0/3512449740 >> v1:172.21.15.23:6789/0 conn(0x7f63ac0c6990 legacy=0x7f63ac0c6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:18.392 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.389+0000 7f63ab7fe700 1 -- 172.21.15.23:0/3512449740 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ac158ce0 con 0x7f63ac06b440 2023-03-27T10:02:18.392 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.389+0000 7f63aa7fc700 1 --2- 172.21.15.23:0/3512449740 >> v2:172.21.15.112:3300/0 conn(0x7f63ac0c9520 0x7f63ac0c8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-27T10:02:18.392 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.390+0000 7f63ab7fe700 1 --2- 172.21.15.23:0/3512449740 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63ac06b440 0x7f63ac06b840 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f63a4009d90 tx=0x7f63a400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6e24fdf6c4de203 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:18.392 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.390+0000 7f63a9ffb700 1 -- 172.21.15.23:0/3512449740 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a400e070 con 0x7f63ac06b440 2023-03-27T10:02:18.392 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.390+0000 7f63a9ffb700 1 -- 172.21.15.23:0/3512449740 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a400bb20 con 0x7f63ac06b440 2023-03-27T10:02:18.392 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.390+0000 7f63a9ffb700 1 -- 172.21.15.23:0/3512449740 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a400be90 con 0x7f63ac06b440 2023-03-27T10:02:18.393 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.390+0000 7f63b15ff700 1 -- 172.21.15.23:0/3512449740 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63ac06b440 msgr2=0x7f63ac06b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:18.393 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.390+0000 7f63b15ff700 1 --2- 172.21.15.23:0/3512449740 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63ac06b440 0x7f63ac06b840 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f63a4009d90 tx=0x7f63a400b330 comp rx=0 tx=0).stop 2023-03-27T10:02:18.393 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.391+0000 7f63b15ff700 1 -- 172.21.15.23:0/3512449740 shutdown_connections 2023-03-27T10:02:18.393 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.391+0000 7f63b15ff700 1 --2- 172.21.15.23:0/3512449740 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63ac06b440 0x7f63ac06b840 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:18.393 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.391+0000 7f63b15ff700 1 --2- 172.21.15.23:0/3512449740 >> v2:172.21.15.112:3300/0 conn(0x7f63ac0c9520 0x7f63ac0c8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:18.393 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.391+0000 7f63b15ff700 1 -- 172.21.15.23:0/3512449740 >> 172.21.15.23:0/3512449740 conn(0x7f63ac071920 msgr2=0x7f63ac071cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:18.393 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.391+0000 7f63b15ff700 1 -- 172.21.15.23:0/3512449740 shutdown_connections 2023-03-27T10:02:18.394 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.391+0000 7f63b15ff700 1 -- 172.21.15.23:0/3512449740 wait complete. 2023-03-27T10:02:18.395 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.392+0000 7f63b15ff700 1 Processor -- start 2023-03-27T10:02:18.395 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.392+0000 7f63b15ff700 1 -- start start 2023-03-27T10:02:18.396 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.393+0000 7f63b15ff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f63ac06b440 0x7f63ac15beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:18.396 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.393+0000 7f63b15ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63ac0c6990 0x7f63ac1ee920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:18.396 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.393+0000 7f63b15ff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63ac0c9520 0x7f63ac157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:18.396 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.393+0000 7f63b15ff700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac1eee60 con 0x7f63ac06b440 2023-03-27T10:02:18.396 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.393+0000 7f63b15ff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac15a180 con 0x7f63ac0c9520 2023-03-27T10:02:18.396 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.393+0000 7f63b15ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac15a440 con 0x7f63ac0c6990 2023-03-27T10:02:18.396 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.393+0000 7f63aaffd700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f63ac06b440 0x7f63ac15beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:18.397 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.394+0000 7f63aaffd700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f63ac06b440 0x7f63ac15beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:34534/0 (socket says 172.21.15.23:34534) 2023-03-27T10:02:18.397 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.394+0000 7f63aaffd700 1 -- 172.21.15.23:0/2523083147 learned_addr learned my addr 172.21.15.23:0/2523083147 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:18.397 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.394+0000 7f63aa7fc700 1 --2- 172.21.15.23:0/2523083147 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63ac0c6990 0x7f63ac1ee920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:18.397 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.394+0000 7f63ab7fe700 1 --2- 172.21.15.23:0/2523083147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63ac0c9520 0x7f63ac157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:18.397 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.394+0000 7f63aaffd700 1 -- 172.21.15.23:0/2523083147 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63ac0c6990 msgr2=0x7f63ac1ee920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:18.397 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.394+0000 7f63aaffd700 1 --2- 172.21.15.23:0/2523083147 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63ac0c6990 0x7f63ac1ee920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:18.397 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.394+0000 7f63aaffd700 1 -- 172.21.15.23:0/2523083147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63ac0c9520 msgr2=0x7f63ac157cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:18.397 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.394+0000 7f63aaffd700 1 --2- 172.21.15.23:0/2523083147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63ac0c9520 0x7f63ac157cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:18.398 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.394+0000 7f63aaffd700 1 -- 172.21.15.23:0/2523083147 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a00094a0 con 0x7f63ac06b440 2023-03-27T10:02:18.398 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.394+0000 7f63aaffd700 1 --2- 172.21.15.23:0/2523083147 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f63ac06b440 0x7f63ac15beb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f639c00c4d0 tx=0x7f639c00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:18.398 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.394+0000 7f63937fe700 1 -- 172.21.15.23:0/2523083147 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639c005d70 con 0x7f63ac06b440 2023-03-27T10:02:18.398 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.394+0000 7f63937fe700 1 -- 172.21.15.23:0/2523083147 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f639c0070d0 con 0x7f63ac06b440 2023-03-27T10:02:18.398 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.394+0000 7f63937fe700 1 -- 172.21.15.23:0/2523083147 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639c007440 con 0x7f63ac06b440 2023-03-27T10:02:18.398 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.394+0000 7f63b15ff700 1 -- 172.21.15.23:0/2523083147 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63a4009a70 con 0x7f63ac06b440 2023-03-27T10:02:18.398 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.395+0000 7f63b15ff700 1 -- 172.21.15.23:0/2523083147 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63ac1592a0 con 0x7f63ac06b440 2023-03-27T10:02:18.399 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.396+0000 7f63b15ff700 1 -- 172.21.15.23:0/2523083147 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63ac15c3f0 con 0x7f63ac06b440 2023-03-27T10:02:18.399 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.396+0000 7f63937fe700 1 -- 172.21.15.23:0/2523083147 <== mon.0 v2:172.21.15.23:3300/0 4 ==== mgrmap(e 6) v1 ==== 85113+0+0 (secure 0 0 0) 0x7f639c019030 con 0x7f63ac06b440 2023-03-27T10:02:18.399 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.397+0000 7f63937fe700 1 --2- 172.21.15.23:0/2523083147 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f6398066e30 0x7f63980692e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:18.400 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.397+0000 7f63937fe700 1 -- 172.21.15.23:0/2523083147 <== mon.0 v2:172.21.15.23:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f639c083750 con 0x7f63ac06b440 2023-03-27T10:02:18.400 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.398+0000 7f63aa7fc700 1 --2- 172.21.15.23:0/2523083147 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f6398066e30 0x7f63980692e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:18.401 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.398+0000 7f63aa7fc700 1 --2- 172.21.15.23:0/2523083147 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f6398066e30 0x7f63980692e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f63ac0cb8f0 tx=0x7f63a0009210 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:18.401 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.402+0000 7f63937fe700 1 -- 172.21.15.23:0/2523083147 <== mon.0 v2:172.21.15.23:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173280 (secure 0 0 0) 0x7f639c04b4f0 con 0x7f63ac06b440 2023-03-27T10:02:18.609 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.610+0000 7f63b15ff700 1 -- 172.21.15.23:0/2523083147 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63ac159580 con 0x7f63ac06b440 2023-03-27T10:02:18.611 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.611+0000 7f63937fe700 1 -- 172.21.15.23:0/2523083147 <== mon.0 v2:172.21.15.23:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+19798 (secure 0 0 0) 0x7f639c050750 con 0x7f63ac06b440 2023-03-27T10:02:18.611 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:02:18.611 INFO:teuthology.orchestra.run.smithi023.stdout:{"epoch":24,"fsid":"1d8d6e6e-0d21-4f51-91ee-d9434d4fb750","created":"2023-03-27T10:01:53.805778+0000","modified":"2023-03-27T10:02:17.274631+0000","last_up_change":"2023-03-27T10:02:04.172266+0000","last_in_change":"2023-03-27T10:01:56.475091+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-27T10:02:05.288883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-27T10:02:05.643442+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-27T10:02:10.720339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-27T10:02:11.254847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-27T10:02:14.809537+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"df086052-536b-4fb1-bcce-7aaf5a13911e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6800","nonce":96642},{"type":"v1","addr":"172.21.15.23:6801","nonce":96642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6802","nonce":96642},{"type":"v1","addr":"172.21.15.23:6803","nonce":96642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6806","nonce":96642},{"type":"v1","addr":"172.21.15.23:6807","nonce":96642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6804","nonce":96642},{"type":"v1","addr":"172.21.15.23:6805","nonce":96642}]},"public_addr":"172.21.15.23:6801/96642","cluster_addr":"172.21.15.23:6803/96642","heartbeat_back_addr":"172.21.15.23:6807/96642","heartbeat_front_addr":"172.21.15.23:6805/96642","state":["exists","up"]},{"osd":1,"uuid":"03138001-a3fd-43fc-9797-a47643d9a3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6808","nonce":96643},{"type":"v1","addr":"172.21.15.23:6809","nonce":96643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6810","nonce":96643},{"type":"v1","addr":"172.21.15.23:6811","nonce":96643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6814","nonce":96643},{"type":"v1","addr":"172.21.15.23:6815","nonce":96643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6812","nonce":96643},{"type":"v1","addr":"172.21.15.23:6813","nonce":96643}]},"public_addr":"172.21.15.23:6809/96643","cluster_addr":"172.21.15.23:6811/96643","heartbeat_back_addr":"172.21.15.23:6815/96643","heartbeat_front_addr":"172.21.15.23:6813/96643","state":["exists","up"]},{"osd":2,"uuid":"964dd926-1160-44ef-aa34-b18965dc05f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6817","nonce":96644},{"type":"v1","addr":"172.21.15.23:6819","nonce":96644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6820","nonce":96644},{"type":"v1","addr":"172.21.15.23:6822","nonce":96644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6827","nonce":96644},{"type":"v1","addr":"172.21.15.23:6829","nonce":96644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6824","nonce":96644},{"type":"v1","addr":"172.21.15.23:6825","nonce":96644}]},"public_addr":"172.21.15.23:6819/96644","cluster_addr":"172.21.15.23:6822/96644","heartbeat_back_addr":"172.21.15.23:6829/96644","heartbeat_front_addr":"172.21.15.23:6825/96644","state":["exists","up"]},{"osd":3,"uuid":"4ae5500f-cb24-44a8-a536-5c399c09e566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6816","nonce":96645},{"type":"v1","addr":"172.21.15.23:6818","nonce":96645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6821","nonce":96645},{"type":"v1","addr":"172.21.15.23:6823","nonce":96645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6830","nonce":96645},{"type":"v1","addr":"172.21.15.23:6831","nonce":96645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6826","nonce":96645},{"type":"v1","addr":"172.21.15.23:6828","nonce":96645}]},"public_addr":"172.21.15.23:6818/96645","cluster_addr":"172.21.15.23:6823/96645","heartbeat_back_addr":"172.21.15.23:6831/96645","heartbeat_front_addr":"172.21.15.23:6828/96645","state":["exists","up"]},{"osd":4,"uuid":"de228ab2-92b4-40bc-8dbd-8f2f4895dc22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":127986},{"type":"v1","addr":"172.21.15.112:6809","nonce":127986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":127986},{"type":"v1","addr":"172.21.15.112:6811","nonce":127986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6815","nonce":127986},{"type":"v1","addr":"172.21.15.112:6818","nonce":127986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":127986},{"type":"v1","addr":"172.21.15.112:6813","nonce":127986}]},"public_addr":"172.21.15.112:6809/127986","cluster_addr":"172.21.15.112:6811/127986","heartbeat_back_addr":"172.21.15.112:6818/127986","heartbeat_front_addr":"172.21.15.112:6813/127986","state":["exists","up"]},{"osd":5,"uuid":"6c839571-b490-4e66-92ac-0d7301c2e677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":127987},{"type":"v1","addr":"172.21.15.112:6801","nonce":127987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":127987},{"type":"v1","addr":"172.21.15.112:6803","nonce":127987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":127987},{"type":"v1","addr":"172.21.15.112:6807","nonce":127987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":127987},{"type":"v1","addr":"172.21.15.112:6805","nonce":127987}]},"public_addr":"172.21.15.112:6801/127987","cluster_addr":"172.21.15.112:6803/127987","heartbeat_back_addr":"172.21.15.112:6807/127987","heartbeat_front_addr":"172.21.15.112:6805/127987","state":["exists","up"]},{"osd":6,"uuid":"871b1a2b-f0ed-4faf-8f9a-a5a0973e11c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":127988},{"type":"v1","addr":"172.21.15.112:6816","nonce":127988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":127988},{"type":"v1","addr":"172.21.15.112:6819","nonce":127988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":127988},{"type":"v1","addr":"172.21.15.112:6823","nonce":127988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":127988},{"type":"v1","addr":"172.21.15.112:6821","nonce":127988}]},"public_addr":"172.21.15.112:6816/127988","cluster_addr":"172.21.15.112:6819/127988","heartbeat_back_addr":"172.21.15.112:6823/127988","heartbeat_front_addr":"172.21.15.112:6821/127988","state":["exists","up"]},{"osd":7,"uuid":"d8228353-34b5-4363-84d8-642512f5bc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":128009},{"type":"v1","addr":"172.21.15.112:6825","nonce":128009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":128009},{"type":"v1","addr":"172.21.15.112:6827","nonce":128009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":128009},{"type":"v1","addr":"172.21.15.112:6831","nonce":128009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":128009},{"type":"v1","addr":"172.21.15.112:6829","nonce":128009}]},"public_addr":"172.21.15.112:6825/128009","cluster_addr":"172.21.15.112:6827/128009","heartbeat_back_addr":"172.21.15.112:6831/128009","heartbeat_front_addr":"172.21.15.112:6829/128009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:02.132160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:01.882517+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:01.927673+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:01.867525+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:01.723580+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:01.730394+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:01.716687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:01.744322+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-27T10:02:18.613 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.614+0000 7f63b15ff700 1 -- 172.21.15.23:0/2523083147 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f6398066e30 msgr2=0x7f63980692e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:18.613 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.614+0000 7f63b15ff700 1 --2- 172.21.15.23:0/2523083147 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f6398066e30 0x7f63980692e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f63ac0cb8f0 tx=0x7f63a0009210 comp rx=0 tx=0).stop 2023-03-27T10:02:18.614 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.614+0000 7f63b15ff700 1 -- 172.21.15.23:0/2523083147 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f63ac06b440 msgr2=0x7f63ac15beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:18.614 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.614+0000 7f63b15ff700 1 --2- 172.21.15.23:0/2523083147 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f63ac06b440 0x7f63ac15beb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f639c00c4d0 tx=0x7f639c00c7e0 comp rx=0 tx=0).stop 2023-03-27T10:02:18.614 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.614+0000 7f63b15ff700 1 -- 172.21.15.23:0/2523083147 shutdown_connections 2023-03-27T10:02:18.614 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.614+0000 7f63b15ff700 1 --2- 172.21.15.23:0/2523083147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63ac0c9520 0x7f63ac157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:18.614 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.614+0000 7f63b15ff700 1 --2- 172.21.15.23:0/2523083147 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f63ac06b440 0x7f63ac15beb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:18.614 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.614+0000 7f63b15ff700 1 --2- 172.21.15.23:0/2523083147 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f6398066e30 0x7f63980692e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:18.615 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.614+0000 7f63b15ff700 1 --2- 172.21.15.23:0/2523083147 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63ac0c6990 0x7f63ac1ee920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:18.615 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.614+0000 7f63b15ff700 1 -- 172.21.15.23:0/2523083147 >> 172.21.15.23:0/2523083147 conn(0x7f63ac071920 msgr2=0x7f63ac0cca50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:18.615 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.614+0000 7f63b15ff700 1 -- 172.21.15.23:0/2523083147 shutdown_connections 2023-03-27T10:02:18.615 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.614+0000 7f63b15ff700 1 -- 172.21.15.23:0/2523083147 wait complete. 2023-03-27T10:02:18.648 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-03-27T10:02:18.837 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.837+0000 7f3fe71ff700 1 Processor -- start 2023-03-27T10:02:18.837 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.837+0000 7f3fe71ff700 1 -- start start 2023-03-27T10:02:18.838 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.838+0000 7f3fe71ff700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f3fe00c6990 0x7f3fe00c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:18.839 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.839+0000 7f3fe71ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3fe00c9520 0x7f3fe00c8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:18.839 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.839+0000 7f3fe71ff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fe00ca730 con 0x7f3fe006b440 2023-03-27T10:02:18.839 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.839+0000 7f3fe71ff700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fe00c72a0 con 0x7f3fe00c6990 2023-03-27T10:02:18.839 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.839+0000 7f3fe71ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe00cf9f0 con 0x7f3fe00c9520 2023-03-27T10:02:18.839 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.839+0000 7f3fe4f9b700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f3fe00c6990 0x7f3fe00c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:18.839 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.839+0000 7f3fe4f9b700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f3fe00c6990 0x7f3fe00c6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:34538/0 (socket says 172.21.15.23:34538) 2023-03-27T10:02:18.839 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.839+0000 7f3fe4f9b700 1 -- 172.21.15.23:0/3409867427 learned_addr learned my addr 172.21.15.23:0/3409867427 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:18.840 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.839+0000 7f3fdffff700 1 --2- 172.21.15.23:0/3409867427 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3fe00c9520 0x7f3fe00c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:18.840 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.839+0000 7f3fe4f9b700 1 -- 172.21.15.23:0/3409867427 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3fe00c9520 msgr2=0x7f3fe00c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:02:18.840 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.839+0000 7f3fe4f9b700 1 --2- 172.21.15.23:0/3409867427 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3fe00c9520 0x7f3fe00c8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:18.840 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.839+0000 7f3fe4f9b700 1 -- 172.21.15.23:0/3409867427 >> v1:172.21.15.112:6789/0 conn(0x7f3fe006b440 legacy=0x7f3fe006b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:18.840 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.839+0000 7f3fe4f9b700 1 -- 172.21.15.23:0/3409867427 --> v2:172.21.15.23:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe0158a00 con 0x7f3fe00c6990 2023-03-27T10:02:18.841 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.840+0000 7f3fe4f9b700 1 --2- 172.21.15.23:0/3409867427 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f3fe00c6990 0x7f3fe00c6d60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3fd40098f0 tx=0x7f3fd4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f6521f7d39f2223 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:18.842 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.840+0000 7f3fdf7fe700 1 -- 172.21.15.23:0/3409867427 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4011830 con 0x7f3fe00c6990 2023-03-27T10:02:18.842 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.840+0000 7f3fdf7fe700 1 -- 172.21.15.23:0/3409867427 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd4011990 con 0x7f3fe00c6990 2023-03-27T10:02:18.842 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.840+0000 7f3fdf7fe700 1 -- 172.21.15.23:0/3409867427 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4011d20 con 0x7f3fe00c6990 2023-03-27T10:02:18.842 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.840+0000 7f3fe71ff700 1 -- 172.21.15.23:0/3409867427 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f3fe00c6990 msgr2=0x7f3fe00c6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:18.842 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.840+0000 7f3fe71ff700 1 --2- 172.21.15.23:0/3409867427 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f3fe00c6990 0x7f3fe00c6d60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3fd40098f0 tx=0x7f3fd4009c00 comp rx=0 tx=0).stop 2023-03-27T10:02:18.842 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.840+0000 7f3fe71ff700 1 -- 172.21.15.23:0/3409867427 shutdown_connections 2023-03-27T10:02:18.842 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.840+0000 7f3fe71ff700 1 --2- 172.21.15.23:0/3409867427 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f3fe00c6990 0x7f3fe00c6d60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:18.843 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.840+0000 7f3fe71ff700 1 --2- 172.21.15.23:0/3409867427 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3fe00c9520 0x7f3fe00c8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:18.843 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.840+0000 7f3fe71ff700 1 -- 172.21.15.23:0/3409867427 >> 172.21.15.23:0/3409867427 conn(0x7f3fe0071920 msgr2=0x7f3fe0071cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:18.843 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.841+0000 7f3fe71ff700 1 -- 172.21.15.23:0/3409867427 shutdown_connections 2023-03-27T10:02:18.843 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.841+0000 7f3fe71ff700 1 -- 172.21.15.23:0/3409867427 wait complete. 2023-03-27T10:02:18.843 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.842+0000 7f3fe71ff700 1 Processor -- start 2023-03-27T10:02:18.843 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.842+0000 7f3fe71ff700 1 -- start start 2023-03-27T10:02:18.845 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.843+0000 7f3fe71ff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fe006b440 0x7f3fe014ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:18.845 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.843+0000 7f3fe71ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3fe00c6990 0x7f3fe01559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:18.845 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.843+0000 7f3fe71ff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f3fe00c9520 0x7f3fe01f1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:18.845 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.843+0000 7f3fe71ff700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe014f710 con 0x7f3fe00c9520 2023-03-27T10:02:18.845 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.843+0000 7f3fe71ff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe0157b50 con 0x7f3fe006b440 2023-03-27T10:02:18.845 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.843+0000 7f3fe71ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe01f1750 con 0x7f3fe00c6990 2023-03-27T10:02:18.846 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.843+0000 7f3fe579c700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f3fe00c9520 0x7f3fe01f1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:18.846 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.843+0000 7f3fe579c700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f3fe00c9520 0x7f3fe01f1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:34542/0 (socket says 172.21.15.23:34542) 2023-03-27T10:02:18.846 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.843+0000 7f3fe579c700 1 -- 172.21.15.23:0/3813590294 learned_addr learned my addr 172.21.15.23:0/3813590294 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:18.846 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.843+0000 7f3fe4f9b700 1 --2- 172.21.15.23:0/3813590294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fe006b440 0x7f3fe014ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:18.846 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.843+0000 7f3fdffff700 1 --2- 172.21.15.23:0/3813590294 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3fe00c6990 0x7f3fe01559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:18.846 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.844+0000 7f3fe579c700 1 -- 172.21.15.23:0/3813590294 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3fe00c6990 msgr2=0x7f3fe01559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:18.846 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.844+0000 7f3fe579c700 1 --2- 172.21.15.23:0/3813590294 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3fe00c6990 0x7f3fe01559f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:18.846 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.844+0000 7f3fe579c700 1 -- 172.21.15.23:0/3813590294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fe006b440 msgr2=0x7f3fe014ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:18.847 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.844+0000 7f3fe579c700 1 --2- 172.21.15.23:0/3813590294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fe006b440 0x7f3fe014ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:18.847 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.844+0000 7f3fe579c700 1 -- 172.21.15.23:0/3813590294 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd4009550 con 0x7f3fe00c9520 2023-03-27T10:02:18.847 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.844+0000 7f3fe4f9b700 1 --2- 172.21.15.23:0/3813590294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fe006b440 0x7f3fe014ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-27T10:02:18.847 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.844+0000 7f3fe579c700 1 --2- 172.21.15.23:0/3813590294 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f3fe00c9520 0x7f3fe01f1ba0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3fd800cc50 tx=0x7f3fd800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:18.847 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.844+0000 7f3fdd7fa700 1 -- 172.21.15.23:0/3813590294 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8003c00 con 0x7f3fe00c9520 2023-03-27T10:02:18.847 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.844+0000 7f3fdd7fa700 1 -- 172.21.15.23:0/3813590294 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd8003d60 con 0x7f3fe00c9520 2023-03-27T10:02:18.847 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.844+0000 7f3fe71ff700 1 -- 172.21.15.23:0/3813590294 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe0158fc0 con 0x7f3fe00c9520 2023-03-27T10:02:18.848 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.844+0000 7f3fdd7fa700 1 -- 172.21.15.23:0/3813590294 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd800aab0 con 0x7f3fe00c9520 2023-03-27T10:02:18.848 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.844+0000 7f3fe71ff700 1 -- 172.21.15.23:0/3813590294 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe0158ce0 con 0x7f3fe00c9520 2023-03-27T10:02:18.848 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.845+0000 7f3fe71ff700 1 -- 172.21.15.23:0/3813590294 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fe01592a0 con 0x7f3fe00c9520 2023-03-27T10:02:18.848 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.846+0000 7f3fdd7fa700 1 -- 172.21.15.23:0/3813590294 <== mon.0 v2:172.21.15.23:3300/0 4 ==== mgrmap(e 6) v1 ==== 85113+0+0 (secure 0 0 0) 0x7f3fd8010050 con 0x7f3fe00c9520 2023-03-27T10:02:18.849 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.847+0000 7f3fdd7fa700 1 --2- 172.21.15.23:0/3813590294 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f3fc4066e30 0x7f3fc40692e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:18.849 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.847+0000 7f3fdd7fa700 1 -- 172.21.15.23:0/3813590294 <== mon.0 v2:172.21.15.23:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3fd8084420 con 0x7f3fe00c9520 2023-03-27T10:02:18.849 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.847+0000 7f3fe4f9b700 1 --2- 172.21.15.23:0/3813590294 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f3fc4066e30 0x7f3fc40692e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:18.849 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.848+0000 7f3fe4f9b700 1 --2- 172.21.15.23:0/3813590294 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f3fc4066e30 0x7f3fc40692e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4009c40 tx=0x7f3fd4008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:18.850 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:18.851+0000 7f3fdd7fa700 1 -- 172.21.15.23:0/3813590294 <== mon.0 v2:172.21.15.23:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173280 (secure 0 0 0) 0x7f3fd8051310 con 0x7f3fe00c9520 2023-03-27T10:02:19.095 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.096+0000 7f3fe71ff700 1 -- 172.21.15.23:0/3813590294 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3fe01f18c0 con 0x7f3fe00c9520 2023-03-27T10:02:19.096 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.097+0000 7f3fdd7fa700 1 -- 172.21.15.23:0/3813590294 <== mon.0 v2:172.21.15.23:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+6126 (secure 0 0 0) 0x7f3fe01f18c0 con 0x7f3fe00c9520 2023-03-27T10:02:19.096 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:02:19.096 INFO:teuthology.orchestra.run.smithi023.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.23:6836/4002126418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6834","nonce":4002126418},{"type":"v1","addr":"172.21.15.23:6836","nonce":4002126418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4253,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.23:6837/1693839903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6835","nonce":1693839903},{"type":"v1","addr":"172.21.15.23:6837","nonce":1693839903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4294,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/357813579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":357813579},{"type":"v1","addr":"172.21.15.112:6836","nonce":357813579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4297,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/394077155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":394077155},{"type":"v1","addr":"172.21.15.112:6837","nonce":394077155}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4300,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/538239518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":538239518},{"type":"v1","addr":"172.21.15.112:6839","nonce":538239518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-03-27T10:02:12.325598+0000","modified":"2023-03-27T10:02:18.276455+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.23:6833/1046971212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6832","nonce":1046971212},{"type":"v1","addr":"172.21.15.23:6833","nonce":1046971212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-03-27T10:02:19.098 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.099+0000 7f3fe71ff700 1 -- 172.21.15.23:0/3813590294 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f3fc4066e30 msgr2=0x7f3fc40692e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:19.098 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.099+0000 7f3fe71ff700 1 --2- 172.21.15.23:0/3813590294 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f3fc4066e30 0x7f3fc40692e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4009c40 tx=0x7f3fd4008030 comp rx=0 tx=0).stop 2023-03-27T10:02:19.099 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.099+0000 7f3fe71ff700 1 -- 172.21.15.23:0/3813590294 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f3fe00c9520 msgr2=0x7f3fe01f1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:19.099 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.099+0000 7f3fe71ff700 1 --2- 172.21.15.23:0/3813590294 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f3fe00c9520 0x7f3fe01f1ba0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3fd800cc50 tx=0x7f3fd800cf60 comp rx=0 tx=0).stop 2023-03-27T10:02:19.099 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.099+0000 7f3fe71ff700 1 -- 172.21.15.23:0/3813590294 shutdown_connections 2023-03-27T10:02:19.099 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.099+0000 7f3fe71ff700 1 --2- 172.21.15.23:0/3813590294 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f3fe00c9520 0x7f3fe01f1ba0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:19.099 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.099+0000 7f3fe71ff700 1 --2- 172.21.15.23:0/3813590294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fe006b440 0x7f3fe014ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:19.099 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.099+0000 7f3fe71ff700 1 --2- 172.21.15.23:0/3813590294 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f3fc4066e30 0x7f3fc40692e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:19.099 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.099+0000 7f3fe71ff700 1 --2- 172.21.15.23:0/3813590294 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3fe00c6990 0x7f3fe01559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:19.100 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.099+0000 7f3fe71ff700 1 -- 172.21.15.23:0/3813590294 >> 172.21.15.23:0/3813590294 conn(0x7f3fe0071920 msgr2=0x7f3fe00c8600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:19.100 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.099+0000 7f3fe71ff700 1 -- 172.21.15.23:0/3813590294 shutdown_connections 2023-03-27T10:02:19.100 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.099+0000 7f3fe71ff700 1 -- 172.21.15.23:0/3813590294 wait complete. 2023-03-27T10:02:19.100 INFO:teuthology.orchestra.run.smithi023.stderr:dumped fsmap epoch 7 2023-03-27T10:02:19.721 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-03-27T10:02:12.325598+0000', 'modified': '2023-03-27T10:02:18.276455+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4306}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4306': {'gid': 4306, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.23:6833/1046971212', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.23:6832', 'nonce': 1046971212}, {'type': 'v1', 'addr': '172.21.15.23:6833', 'nonce': 1046971212}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-03-27T10:02:19.722 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-03-27T10:02:19.722 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-03-27T10:02:19.918 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.919+0000 7f99ff3ff700 1 Processor -- start 2023-03-27T10:02:19.919 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.919+0000 7f99ff3ff700 1 -- start start 2023-03-27T10:02:19.919 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.920+0000 7f99ff3ff700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f99f80c6920 0x7f99f806f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:19.920 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.920+0000 7f99ff3ff700 1 -- --> v1:172.21.15.23:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99f806e820 con 0x7f99f80c7320 2023-03-27T10:02:19.920 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.920+0000 7f99ff3ff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99f80c86a0 con 0x7f99f806b300 2023-03-27T10:02:19.920 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.920+0000 7f99ff3ff700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99f8159690 con 0x7f99f80c6920 2023-03-27T10:02:19.921 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.920+0000 7f99fd19b700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f99f80c6920 0x7f99f806f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:19.922 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.921+0000 7f99fd19b700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f99f80c6920 0x7f99f806f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:34548/0 (socket says 172.21.15.23:34548) 2023-03-27T10:02:19.922 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.921+0000 7f99fd19b700 1 -- 172.21.15.23:0/314854238 learned_addr learned my addr 172.21.15.23:0/314854238 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:19.922 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.921+0000 7f99fc99a700 1 --1- 172.21.15.23:0/314854238 >> v1:172.21.15.23:6789/0 conn(0x7f99f80c7320 0x7f99f80c7f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.23:6789/0 says I am v1:172.21.15.23:36732/0 (socket says 172.21.15.23:36732) 2023-03-27T10:02:19.922 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.921+0000 7f99fd19b700 1 -- 172.21.15.23:0/314854238 >> v1:172.21.15.112:6789/0 conn(0x7f99f806b300 legacy=0x7f99f8163150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:19.922 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.921+0000 7f99fd19b700 1 -- 172.21.15.23:0/314854238 >> v1:172.21.15.23:6789/0 conn(0x7f99f80c7320 legacy=0x7f99f80c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:19.922 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.921+0000 7f99fd19b700 1 -- 172.21.15.23:0/314854238 --> v2:172.21.15.23:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f806dc10 con 0x7f99f80c6920 2023-03-27T10:02:19.922 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.921+0000 7f99fd19b700 1 --2- 172.21.15.23:0/314854238 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f99f80c6920 0x7f99f806f200 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f99f40098f0 tx=0x7f99f4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a76702f61a56b16 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:19.922 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.921+0000 7f99effff700 1 -- 172.21.15.23:0/314854238 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f400b8a0 con 0x7f99f80c6920 2023-03-27T10:02:19.923 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.921+0000 7f99effff700 1 -- 172.21.15.23:0/314854238 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f400ba00 con 0x7f99f80c6920 2023-03-27T10:02:19.923 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.922+0000 7f99effff700 1 -- 172.21.15.23:0/314854238 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f400bd90 con 0x7f99f80c6920 2023-03-27T10:02:19.923 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.922+0000 7f99ff3ff700 1 -- 172.21.15.23:0/314854238 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f99f80c6920 msgr2=0x7f99f806f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:19.923 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.922+0000 7f99ff3ff700 1 --2- 172.21.15.23:0/314854238 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f99f80c6920 0x7f99f806f200 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f99f40098f0 tx=0x7f99f4009c00 comp rx=0 tx=0).stop 2023-03-27T10:02:19.923 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.922+0000 7f99ff3ff700 1 -- 172.21.15.23:0/314854238 shutdown_connections 2023-03-27T10:02:19.923 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.922+0000 7f99ff3ff700 1 --2- 172.21.15.23:0/314854238 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f99f80c6920 0x7f99f806f200 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:19.923 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.922+0000 7f99ff3ff700 1 -- 172.21.15.23:0/314854238 >> 172.21.15.23:0/314854238 conn(0x7f99f8065ca0 msgr2=0x7f99f8066070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:19.924 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.922+0000 7f99ff3ff700 1 -- 172.21.15.23:0/314854238 shutdown_connections 2023-03-27T10:02:19.924 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.922+0000 7f99ff3ff700 1 -- 172.21.15.23:0/314854238 wait complete. 2023-03-27T10:02:19.925 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.924+0000 7f99ff3ff700 1 Processor -- start 2023-03-27T10:02:19.925 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.924+0000 7f99ff3ff700 1 -- start start 2023-03-27T10:02:19.925 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.925+0000 7f99ff3ff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99f806b300 0x7f99f81518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:19.925 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.925+0000 7f99ff3ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f99f80c6920 0x7f99f81ee3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:19.925 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.925+0000 7f99ff3ff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f99f80c7320 0x7f99f81ee8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:19.925 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.925+0000 7f99ff3ff700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f814fbf0 con 0x7f99f80c7320 2023-03-27T10:02:19.926 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.925+0000 7f99ff3ff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f81f2490 con 0x7f99f806b300 2023-03-27T10:02:19.926 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.925+0000 7f99ff3ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f81f1910 con 0x7f99f80c6920 2023-03-27T10:02:19.926 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.925+0000 7f99fd19b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99f806b300 0x7f99f81518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:19.926 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.925+0000 7f99fd19b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99f806b300 0x7f99f81518c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.23:47984/0 (socket says 172.21.15.23:47984) 2023-03-27T10:02:19.926 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.925+0000 7f99fd19b700 1 -- 172.21.15.23:0/1319374964 learned_addr learned my addr 172.21.15.23:0/1319374964 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:19.926 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.925+0000 7f99fd99c700 1 --2- 172.21.15.23:0/1319374964 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f99f80c7320 0x7f99f81ee8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:19.926 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.925+0000 7f99fc99a700 1 --2- 172.21.15.23:0/1319374964 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f99f80c6920 0x7f99f81ee3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:19.927 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.926+0000 7f99fd19b700 1 -- 172.21.15.23:0/1319374964 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f99f80c6920 msgr2=0x7f99f81ee3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:19.928 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.926+0000 7f99fd19b700 1 --2- 172.21.15.23:0/1319374964 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f99f80c6920 0x7f99f81ee3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:19.928 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.926+0000 7f99fd19b700 1 -- 172.21.15.23:0/1319374964 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f99f80c7320 msgr2=0x7f99f81ee8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:19.928 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.926+0000 7f99fd19b700 1 --2- 172.21.15.23:0/1319374964 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f99f80c7320 0x7f99f81ee8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:19.928 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.926+0000 7f99fd19b700 1 -- 172.21.15.23:0/1319374964 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f4009550 con 0x7f99f806b300 2023-03-27T10:02:19.928 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.926+0000 7f99fd99c700 1 --2- 172.21.15.23:0/1319374964 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f99f80c7320 0x7f99f81ee8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-27T10:02:19.928 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.926+0000 7f99fd19b700 1 --2- 172.21.15.23:0/1319374964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99f806b300 0x7f99f81518c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f99f4009c40 tx=0x7f99f4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:19.928 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.926+0000 7f99edffb700 1 -- 172.21.15.23:0/1319374964 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f4004380 con 0x7f99f806b300 2023-03-27T10:02:19.928 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.926+0000 7f99edffb700 1 -- 172.21.15.23:0/1319374964 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f401f070 con 0x7f99f806b300 2023-03-27T10:02:19.929 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.926+0000 7f99ff3ff700 1 -- 172.21.15.23:0/1319374964 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f806e1d0 con 0x7f99f806b300 2023-03-27T10:02:19.929 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.926+0000 7f99edffb700 1 -- 172.21.15.23:0/1319374964 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f40036a0 con 0x7f99f806b300 2023-03-27T10:02:19.929 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.926+0000 7f99ff3ff700 1 -- 172.21.15.23:0/1319374964 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f806def0 con 0x7f99f806b300 2023-03-27T10:02:19.930 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.929+0000 7f99edffb700 1 -- 172.21.15.23:0/1319374964 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 85113+0+0 (secure 0 0 0) 0x7f99f402a020 con 0x7f99f806b300 2023-03-27T10:02:19.930 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.929+0000 7f99edffb700 1 --2- 172.21.15.23:0/1319374964 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f99dc066e30 0x7f99dc0692e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:19.930 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.929+0000 7f99edffb700 1 -- 172.21.15.23:0/1319374964 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f99f4085a50 con 0x7f99f806b300 2023-03-27T10:02:19.930 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.930+0000 7f99fc99a700 1 --2- 172.21.15.23:0/1319374964 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f99dc066e30 0x7f99dc0692e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:19.930 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.930+0000 7f99fc99a700 1 --2- 172.21.15.23:0/1319374964 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f99dc066e30 0x7f99dc0692e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f99f81568a0 tx=0x7f99e8007ed0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:19.931 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.931+0000 7f9a12d95b80 1 Processor -- start 2023-03-27T10:02:19.931 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.931+0000 7f9a12d95b80 1 -- start start 2023-03-27T10:02:19.932 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.932+0000 7f9a12d95b80 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c87b3c28d0 0x55c87b40ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:19.932 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.932+0000 7f9a12d95b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c87b402550 0x55c87b3bb160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:19.932 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.932+0000 7f9a12d95b80 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x55c87b3e9d60 0x55c87b3bb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:19.932 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.932+0000 7f9a12d95b80 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c87b2ab950 con 0x55c87b3e9d60 2023-03-27T10:02:19.933 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.932+0000 7f9a12d95b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c87b31a0d0 con 0x55c87b402550 2023-03-27T10:02:19.933 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.932+0000 7f9a12d95b80 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c87b313600 con 0x55c87b3c28d0 2023-03-27T10:02:19.933 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.932+0000 7f99fc99a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c87b402550 0x55c87b3bb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:19.933 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.932+0000 7f99fd99c700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x55c87b3e9d60 0x55c87b3bb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:19.933 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.932+0000 7f99fc99a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c87b402550 0x55c87b3bb160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.23:47994/0 (socket says 172.21.15.23:47994) 2023-03-27T10:02:19.933 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.932+0000 7f99fd19b700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c87b3c28d0 0x55c87b40ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:19.933 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.932+0000 7f99fc99a700 1 -- 172.21.15.23:0/1096959722 learned_addr learned my addr 172.21.15.23:0/1096959722 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:19.934 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.933+0000 7f99fd19b700 1 -- 172.21.15.23:0/1096959722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c87b402550 msgr2=0x55c87b3bb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:02:19.934 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.933+0000 7f99fd19b700 1 --2- 172.21.15.23:0/1096959722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c87b402550 0x55c87b3bb160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:19.934 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.933+0000 7f99fd19b700 1 -- 172.21.15.23:0/1096959722 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x55c87b3e9d60 msgr2=0x55c87b3bb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:02:19.934 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.933+0000 7f99fd19b700 1 --2- 172.21.15.23:0/1096959722 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x55c87b3e9d60 0x55c87b3bb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:19.934 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.933+0000 7f99fd19b700 1 -- 172.21.15.23:0/1096959722 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c87b3764a0 con 0x55c87b3c28d0 2023-03-27T10:02:19.934 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.933+0000 7f99fd19b700 1 --2- 172.21.15.23:0/1096959722 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c87b3c28d0 0x55c87b40ab00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f99f4056540 tx=0x7f99f4056040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=417a0416dd5dfbc3 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:19.937 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.933+0000 7f99e2ffd700 1 -- 172.21.15.23:0/1096959722 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f4080020 con 0x55c87b3c28d0 2023-03-27T10:02:19.937 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.933+0000 7f99e2ffd700 1 -- 172.21.15.23:0/1096959722 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f40193b0 con 0x55c87b3c28d0 2023-03-27T10:02:19.937 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.933+0000 7f99e2ffd700 1 -- 172.21.15.23:0/1096959722 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f4055b50 con 0x55c87b3c28d0 2023-03-27T10:02:19.937 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.934+0000 7f9a12d95b80 1 -- 172.21.15.23:0/1096959722 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c87b3c28d0 msgr2=0x55c87b40ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:19.937 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.934+0000 7f9a12d95b80 1 --2- 172.21.15.23:0/1096959722 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c87b3c28d0 0x55c87b40ab00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f99f4056540 tx=0x7f99f4056040 comp rx=0 tx=0).stop 2023-03-27T10:02:19.938 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.934+0000 7f9a12d95b80 1 -- 172.21.15.23:0/1096959722 shutdown_connections 2023-03-27T10:02:19.938 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.934+0000 7f9a12d95b80 1 --2- 172.21.15.23:0/1096959722 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x55c87b3e9d60 0x55c87b3bb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:19.938 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.934+0000 7f9a12d95b80 1 --2- 172.21.15.23:0/1096959722 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c87b3c28d0 0x55c87b40ab00 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:19.938 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.934+0000 7f9a12d95b80 1 --2- 172.21.15.23:0/1096959722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c87b402550 0x55c87b3bb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:19.938 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.934+0000 7f9a12d95b80 1 -- 172.21.15.23:0/1096959722 >> 172.21.15.23:0/1096959722 conn(0x55c87b3bda40 msgr2=0x55c87b3be830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:19.938 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.934+0000 7f9a12d95b80 1 -- 172.21.15.23:0/1096959722 shutdown_connections 2023-03-27T10:02:19.938 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.934+0000 7f9a12d95b80 1 -- 172.21.15.23:0/1096959722 wait complete. 2023-03-27T10:02:19.939 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.935+0000 7f9a12d95b80 1 -- start start 2023-03-27T10:02:19.939 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.935+0000 7f9a12d95b80 1 Processor -- start 2023-03-27T10:02:19.939 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.936+0000 7f9a12d95b80 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c87b3c28d0 0x55c87b3e5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:19.939 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.936+0000 7f9a12d95b80 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x55c87b3e9d60 0x55c87b411ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:19.939 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.936+0000 7f9a12d95b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c87b402950 0x55c87b3e64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:19.939 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.936+0000 7f9a12d95b80 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c87b1978d0 con 0x55c87b3e9d60 2023-03-27T10:02:19.939 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.936+0000 7f9a12d95b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c87b3e5ca0 con 0x55c87b402950 2023-03-27T10:02:19.939 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.936+0000 7f9a12d95b80 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c87b4030c0 con 0x55c87b3c28d0 2023-03-27T10:02:19.940 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.936+0000 7f9a12d95b80 10 client.0 _ll_register_callbacks cb 0x55c87b2b94d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-03-27T10:02:19.940 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.936+0000 7f99fd19b700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c87b3c28d0 0x55c87b3e5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:19.940 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.936+0000 7f99fc99a700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x55c87b3e9d60 0x55c87b411ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:19.940 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.936+0000 7f99fd19b700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c87b3c28d0 0x55c87b3e5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.23:43786/0 (socket says 172.21.15.23:43786) 2023-03-27T10:02:19.940 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.936+0000 7f99fd19b700 1 -- 172.21.15.23:0/3957177241 learned_addr learned my addr 172.21.15.23:0/3957177241 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:19.940 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.936+0000 7f99fd99c700 1 --2- 172.21.15.23:0/3957177241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c87b402950 0x55c87b3e64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:19.940 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.936+0000 7f99fd19b700 1 -- 172.21.15.23:0/3957177241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c87b402950 msgr2=0x55c87b3e64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:02:19.940 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.936+0000 7f99fd19b700 1 --2- 172.21.15.23:0/3957177241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c87b402950 0x55c87b3e64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:19.941 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.936+0000 7f99fd19b700 1 -- 172.21.15.23:0/3957177241 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x55c87b3e9d60 msgr2=0x55c87b411ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:02:19.941 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.936+0000 7f99fd19b700 1 --2- 172.21.15.23:0/3957177241 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x55c87b3e9d60 0x55c87b411ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:19.941 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.936+0000 7f99fd19b700 1 -- 172.21.15.23:0/3957177241 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f806def0 con 0x55c87b3c28d0 2023-03-27T10:02:19.941 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.937+0000 7f99fd19b700 1 --2- 172.21.15.23:0/3957177241 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c87b3c28d0 0x55c87b3e5760 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f99f40549b0 tx=0x7f99f4059070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3b942a4a0ab2cc6 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:19.941 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.937+0000 7f99d7fff700 10 client.0 ms_handle_connect on v2:172.21.15.112:3301/0 2023-03-27T10:02:19.941 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.937+0000 7f99d7fff700 1 -- 172.21.15.23:0/3957177241 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f4080040 con 0x55c87b3c28d0 2023-03-27T10:02:19.941 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.937+0000 7f99d7fff700 1 -- 172.21.15.23:0/3957177241 <== mon.2 v2:172.21.15.112:3301/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f99f4055b50 con 0x55c87b3c28d0 2023-03-27T10:02:19.942 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.937+0000 7f9a12d95b80 1 -- 172.21.15.23:0/3957177241 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c87b2d5240 con 0x55c87b3c28d0 2023-03-27T10:02:19.942 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.937+0000 7f99d7fff700 1 -- 172.21.15.23:0/3957177241 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99f40195b0 con 0x55c87b3c28d0 2023-03-27T10:02:19.942 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.937+0000 7f9a12d95b80 10 client.4355 fetch_fsmap learned FSMap version 7 2023-03-27T10:02:19.942 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.937+0000 7f9a12d95b80 1 -- 172.21.15.23:0/3957177241 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55c87b141d90 con 0x55c87b3c28d0 2023-03-27T10:02:19.942 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.937+0000 7f99d7fff700 1 -- 172.21.15.23:0/3957177241 <== mon.2 v2:172.21.15.112:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c87b2d5240 con 0x55c87b3c28d0 2023-03-27T10:02:19.942 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.937+0000 7f99d7fff700 1 -- 172.21.15.23:0/3957177241 <== mon.2 v2:172.21.15.112:3301/0 5 ==== fsmap(e 7) v1 ==== 4204+0+0 (secure 0 0 0) 0x7f99f4065030 con 0x55c87b3c28d0 2023-03-27T10:02:19.942 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.937+0000 7f9a12d95b80 10 client.4355 fetch_fsmap finished waiting for FSMap version 7 2023-03-27T10:02:19.942 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.937+0000 7f9a12d95b80 10 client.4355 resolve_mds: resolved name 'a' to daemon mds.a 2023-03-27T10:02:19.943 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.937+0000 7f9a12d95b80 20 client.4355 populate_metadata read hostname 'smithi023' 2023-03-27T10:02:19.943 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.937+0000 7f9a12d95b80 1 --2- 172.21.15.23:0/3957177241 >> [v2:172.21.15.23:6832/1046971212,v1:172.21.15.23:6833/1046971212] conn(0x55c87b414790 0x55c87b414b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:19.943 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.937+0000 7f9a12d95b80 4 client.4355 mds_command: new command op to 4306 tid=0[{"prefix": "get_command_descriptions"}] 2023-03-27T10:02:19.943 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.937+0000 7f9a12d95b80 1 -- 172.21.15.23:0/3957177241 --> [v2:172.21.15.23:6832/1046971212,v1:172.21.15.23:6833/1046971212] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c87b197f10 con 0x55c87b414790 2023-03-27T10:02:19.943 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.938+0000 7f99fc99a700 1 --2- 172.21.15.23:0/3957177241 >> [v2:172.21.15.23:6832/1046971212,v1:172.21.15.23:6833/1046971212] conn(0x55c87b414790 0x55c87b414b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:19.944 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.938+0000 7f99fc99a700 1 --2- 172.21.15.23:0/3957177241 >> [v2:172.21.15.23:6832/1046971212,v1:172.21.15.23:6833/1046971212] conn(0x55c87b414790 0x55c87b414b60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d238ed2d65d63fc7 server_cookie=9281b817bab5e0c3 in_seq=0 out_seq=0 2023-03-27T10:02:19.944 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:19.938+0000 7f99d7fff700 10 client.4355 ms_handle_connect on v2:172.21.15.23:6832/1046971212 2023-03-27T10:02:23.282 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.282+0000 7f99d7fff700 1 -- 172.21.15.23:0/3957177241 <== mds.0 v2:172.21.15.23:6832/1046971212 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17527 (crc 0 0 0) 0x55c87b197f10 con 0x55c87b414790 2023-03-27T10:02:23.282 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.282+0000 7f99d7fff700 10 client.4355 handle_command_reply: tid=0 2023-03-27T10:02:23.282 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.282+0000 7f9a12d95b80 1 client.4355 shutdown 2023-03-27T10:02:23.283 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.282+0000 7f9a12d95b80 1 -- 172.21.15.23:0/3957177241 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c87b3c28d0 msgr2=0x55c87b3e5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:23.283 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.282+0000 7f9a12d95b80 1 --2- 172.21.15.23:0/3957177241 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c87b3c28d0 0x55c87b3e5760 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f99f40549b0 tx=0x7f99f4059070 comp rx=0 tx=0).stop 2023-03-27T10:02:23.283 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.282+0000 7f9a12d95b80 1 -- 172.21.15.23:0/3957177241 shutdown_connections 2023-03-27T10:02:23.283 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.282+0000 7f9a12d95b80 1 --2- 172.21.15.23:0/3957177241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c87b402950 0x55c87b3e64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:23.284 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.282+0000 7f9a12d95b80 1 --2- 172.21.15.23:0/3957177241 >> [v2:172.21.15.23:6832/1046971212,v1:172.21.15.23:6833/1046971212] conn(0x55c87b414790 0x55c87b414b60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:23.284 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.282+0000 7f9a12d95b80 1 --2- 172.21.15.23:0/3957177241 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c87b3c28d0 0x55c87b3e5760 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:23.284 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.282+0000 7f9a12d95b80 1 --2- 172.21.15.23:0/3957177241 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x55c87b3e9d60 0x55c87b411ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:23.284 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.282+0000 7f99d7fff700 0 client.4355 ms_handle_reset on v2:172.21.15.23:6832/1046971212 2023-03-27T10:02:23.284 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.282+0000 7f9a12d95b80 1 -- 172.21.15.23:0/3957177241 >> 172.21.15.23:0/3957177241 conn(0x55c87b3bda40 msgr2=0x55c87b404f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:23.284 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.283+0000 7f9a12d95b80 1 -- 172.21.15.23:0/3957177241 shutdown_connections 2023-03-27T10:02:23.284 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.283+0000 7f9a12d95b80 1 -- 172.21.15.23:0/3957177241 wait complete. 2023-03-27T10:02:23.285 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.283+0000 7f9a12d95b80 20 client.4355 trim_cache size 0 max 16384 2023-03-27T10:02:23.302 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.302+0000 7f9a12d95b80 1 Processor -- start 2023-03-27T10:02:23.302 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.302+0000 7f9a12d95b80 1 -- start start 2023-03-27T10:02:23.304 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.303+0000 7f9a12d95b80 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c87b3c28d0 0x55c87b3ea940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:23.304 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.303+0000 7f9a12d95b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c87b3e9d60 0x55c87b3bc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:23.304 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.303+0000 7f9a12d95b80 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x55c87b402950 0x55c87b3e8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:23.304 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.303+0000 7f9a12d95b80 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c87b3b77f0 con 0x55c87b402950 2023-03-27T10:02:23.305 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.303+0000 7f9a12d95b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c87b400d40 con 0x55c87b3e9d60 2023-03-27T10:02:23.305 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.303+0000 7f9a12d95b80 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c87b4145d0 con 0x55c87b3c28d0 2023-03-27T10:02:23.305 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.303+0000 7f99fd99c700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x55c87b402950 0x55c87b3e8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:23.305 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.304+0000 7f99fd19b700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c87b3c28d0 0x55c87b3ea940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:23.305 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.304+0000 7f99fc99a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c87b3e9d60 0x55c87b3bc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:23.305 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.304+0000 7f99fd99c700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x55c87b402950 0x55c87b3e8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:34592/0 (socket says 172.21.15.23:34592) 2023-03-27T10:02:23.305 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.304+0000 7f99fd99c700 1 -- 172.21.15.23:0/1888540930 learned_addr learned my addr 172.21.15.23:0/1888540930 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:23.306 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.304+0000 7f99fd19b700 1 -- 172.21.15.23:0/1888540930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c87b3e9d60 msgr2=0x55c87b3bc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:02:23.306 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.304+0000 7f99fd19b700 1 --2- 172.21.15.23:0/1888540930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c87b3e9d60 0x55c87b3bc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:23.306 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.304+0000 7f99fd19b700 1 -- 172.21.15.23:0/1888540930 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x55c87b402950 msgr2=0x55c87b3e8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:02:23.306 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.304+0000 7f99fd19b700 1 --2- 172.21.15.23:0/1888540930 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x55c87b402950 0x55c87b3e8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:23.306 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.304+0000 7f99fd19b700 1 -- 172.21.15.23:0/1888540930 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c87b2da140 con 0x55c87b3c28d0 2023-03-27T10:02:23.307 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.304+0000 7f99fc99a700 1 --2- 172.21.15.23:0/1888540930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c87b3e9d60 0x55c87b3bc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-27T10:02:23.307 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.304+0000 7f99fd19b700 1 --2- 172.21.15.23:0/1888540930 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c87b3c28d0 0x55c87b3ea940 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f99f4009520 tx=0x7f99f405f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b385be1c54f44f13 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:23.307 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.304+0000 7f99e37fe700 1 -- 172.21.15.23:0/1888540930 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f4080070 con 0x55c87b3c28d0 2023-03-27T10:02:23.307 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.304+0000 7f99e37fe700 1 -- 172.21.15.23:0/1888540930 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f4061070 con 0x55c87b3c28d0 2023-03-27T10:02:23.307 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.305+0000 7f99e37fe700 1 -- 172.21.15.23:0/1888540930 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f40195b0 con 0x55c87b3c28d0 2023-03-27T10:02:23.311 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.305+0000 7f9a12d95b80 1 -- 172.21.15.23:0/1888540930 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c87b3c28d0 msgr2=0x55c87b3ea940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:23.311 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.305+0000 7f9a12d95b80 1 --2- 172.21.15.23:0/1888540930 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c87b3c28d0 0x55c87b3ea940 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f99f4009520 tx=0x7f99f405f000 comp rx=0 tx=0).stop 2023-03-27T10:02:23.311 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.305+0000 7f9a12d95b80 1 -- 172.21.15.23:0/1888540930 shutdown_connections 2023-03-27T10:02:23.311 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.305+0000 7f9a12d95b80 1 --2- 172.21.15.23:0/1888540930 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x55c87b402950 0x55c87b3e8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:23.311 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.305+0000 7f9a12d95b80 1 --2- 172.21.15.23:0/1888540930 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c87b3c28d0 0x55c87b3ea940 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:23.312 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.305+0000 7f9a12d95b80 1 --2- 172.21.15.23:0/1888540930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c87b3e9d60 0x55c87b3bc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:23.312 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.305+0000 7f9a12d95b80 1 -- 172.21.15.23:0/1888540930 >> 172.21.15.23:0/1888540930 conn(0x55c87b3bda40 msgr2=0x55c87b3e5570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:23.312 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.305+0000 7f9a12d95b80 1 -- 172.21.15.23:0/1888540930 shutdown_connections 2023-03-27T10:02:23.312 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.306+0000 7f9a12d95b80 1 -- 172.21.15.23:0/1888540930 wait complete. 2023-03-27T10:02:23.312 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.306+0000 7f9a12d95b80 1 -- start start 2023-03-27T10:02:23.312 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.306+0000 7f9a12d95b80 1 Processor -- start 2023-03-27T10:02:23.312 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.307+0000 7f9a12d95b80 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x55c87b3c28d0 0x55c87b3fe3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:23.313 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.308+0000 7f9a12d95b80 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c87b3e9d60 0x55c87b3e5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:23.313 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.308+0000 7f9a12d95b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c87b402950 0x55c87b4601d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:23.313 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.308+0000 7f9a12d95b80 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c87b3fe900 con 0x55c87b3c28d0 2023-03-27T10:02:23.313 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.308+0000 7f9a12d95b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c87b3e5ad0 con 0x55c87b402950 2023-03-27T10:02:23.313 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.308+0000 7f9a12d95b80 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c87b3fed30 con 0x55c87b3e9d60 2023-03-27T10:02:23.313 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.308+0000 7f9a12d95b80 10 client.0 _ll_register_callbacks cb 0x55c87b2b94d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-03-27T10:02:23.313 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.308+0000 7f99fd19b700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x55c87b3c28d0 0x55c87b3fe3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:23.314 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.308+0000 7f99fd99c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c87b402950 0x55c87b4601d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:23.314 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.308+0000 7f99fd19b700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x55c87b3c28d0 0x55c87b3fe3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:34600/0 (socket says 172.21.15.23:34600) 2023-03-27T10:02:23.314 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.308+0000 7f99fd19b700 1 -- 172.21.15.23:0/671482033 learned_addr learned my addr 172.21.15.23:0/671482033 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:23.314 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.308+0000 7f99fc99a700 1 --2- 172.21.15.23:0/671482033 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c87b3e9d60 0x55c87b3e5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:23.314 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.308+0000 7f99fd19b700 1 -- 172.21.15.23:0/671482033 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c87b3e9d60 msgr2=0x55c87b3e5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:02:23.314 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.308+0000 7f99fd19b700 1 --2- 172.21.15.23:0/671482033 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c87b3e9d60 0x55c87b3e5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:23.314 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.308+0000 7f99fd19b700 1 -- 172.21.15.23:0/671482033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c87b402950 msgr2=0x55c87b4601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:02:23.315 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.308+0000 7f99fd19b700 1 --2- 172.21.15.23:0/671482033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c87b402950 0x55c87b4601d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:23.315 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.308+0000 7f99fd19b700 1 -- 172.21.15.23:0/671482033 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f806e820 con 0x55c87b3c28d0 2023-03-27T10:02:23.315 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.308+0000 7f99fd99c700 1 --2- 172.21.15.23:0/671482033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c87b402950 0x55c87b4601d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-27T10:02:23.315 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.309+0000 7f99fd19b700 1 --2- 172.21.15.23:0/671482033 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x55c87b3c28d0 0x55c87b3fe3c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f99f4004090 tx=0x7f99f4052a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7615121aedb178 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:23.315 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.309+0000 7f99d7fff700 10 client.0 ms_handle_connect on v2:172.21.15.23:3300/0 2023-03-27T10:02:23.315 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.309+0000 7f99d7fff700 1 -- 172.21.15.23:0/671482033 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f4080040 con 0x55c87b3c28d0 2023-03-27T10:02:23.315 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.309+0000 7f99d7fff700 1 -- 172.21.15.23:0/671482033 <== mon.0 v2:172.21.15.23:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f99f405d790 con 0x55c87b3c28d0 2023-03-27T10:02:23.315 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.309+0000 7f9a12d95b80 1 -- 172.21.15.23:0/671482033 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c87b2cede0 con 0x55c87b3c28d0 2023-03-27T10:02:23.316 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.309+0000 7f99d7fff700 1 -- 172.21.15.23:0/671482033 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99f4055e00 con 0x55c87b3c28d0 2023-03-27T10:02:23.316 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.309+0000 7f9a12d95b80 10 client.4398 fetch_fsmap learned FSMap version 7 2023-03-27T10:02:23.316 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.309+0000 7f9a12d95b80 1 -- 172.21.15.23:0/671482033 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x7f99f806def0 con 0x55c87b3c28d0 2023-03-27T10:02:23.316 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.309+0000 7f99d7fff700 1 -- 172.21.15.23:0/671482033 <== mon.0 v2:172.21.15.23:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c87b2cede0 con 0x55c87b3c28d0 2023-03-27T10:02:23.316 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.310+0000 7f99d7fff700 1 -- 172.21.15.23:0/671482033 <== mon.0 v2:172.21.15.23:3300/0 5 ==== fsmap(e 7) v1 ==== 4204+0+0 (secure 0 0 0) 0x7f99f4052d70 con 0x55c87b3c28d0 2023-03-27T10:02:23.316 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.310+0000 7f9a12d95b80 10 client.4398 fetch_fsmap finished waiting for FSMap version 7 2023-03-27T10:02:23.316 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.310+0000 7f9a12d95b80 10 client.4398 resolve_mds: resolved name 'a' to daemon mds.a 2023-03-27T10:02:23.317 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.310+0000 7f9a12d95b80 20 client.4398 populate_metadata read hostname 'smithi023' 2023-03-27T10:02:23.317 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.310+0000 7f9a12d95b80 1 --2- 172.21.15.23:0/671482033 >> [v2:172.21.15.23:6832/1046971212,v1:172.21.15.23:6833/1046971212] conn(0x55c87b460920 0x55c87b3fa7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:23.317 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.310+0000 7f9a12d95b80 4 client.4398 mds_command: new command op to 4306 tid=0[{"prefix": "status"}] 2023-03-27T10:02:23.317 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.310+0000 7f9a12d95b80 1 -- 172.21.15.23:0/671482033 --> [v2:172.21.15.23:6832/1046971212,v1:172.21.15.23:6833/1046971212] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c87b2b4c80 con 0x55c87b460920 2023-03-27T10:02:23.317 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.310+0000 7f99fc99a700 1 --2- 172.21.15.23:0/671482033 >> [v2:172.21.15.23:6832/1046971212,v1:172.21.15.23:6833/1046971212] conn(0x55c87b460920 0x55c87b3fa7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:23.317 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.311+0000 7f99d7fff700 10 client.4398 ms_handle_connect on v2:172.21.15.23:6832/1046971212 2023-03-27T10:02:23.317 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.311+0000 7f99fc99a700 1 --2- 172.21.15.23:0/671482033 >> [v2:172.21.15.23:6832/1046971212,v1:172.21.15.23:6833/1046971212] conn(0x55c87b460920 0x55c87b3fa7c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=252f7ec68d9ae252 server_cookie=32038eaf470bc3d5 in_seq=0 out_seq=0 2023-03-27T10:02:23.318 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.311+0000 7f99d7fff700 1 -- 172.21.15.23:0/671482033 <== mds.0 v2:172.21.15.23:6832/1046971212 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55c87b2b4c80 con 0x55c87b460920 2023-03-27T10:02:23.318 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.311+0000 7f99d7fff700 10 client.4398 handle_command_reply: tid=0 2023-03-27T10:02:23.318 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.311+0000 7f9a12d95b80 1 client.4398 shutdown 2023-03-27T10:02:23.318 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.311+0000 7f9a12d95b80 1 -- 172.21.15.23:0/671482033 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x55c87b3c28d0 msgr2=0x55c87b3fe3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:23.318 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.311+0000 7f9a12d95b80 1 --2- 172.21.15.23:0/671482033 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x55c87b3c28d0 0x55c87b3fe3c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f99f4004090 tx=0x7f99f4052a20 comp rx=0 tx=0).stop 2023-03-27T10:02:23.318 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.311+0000 7f9a12d95b80 1 -- 172.21.15.23:0/671482033 shutdown_connections 2023-03-27T10:02:23.318 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.311+0000 7f9a12d95b80 1 --2- 172.21.15.23:0/671482033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c87b402950 0x55c87b4601d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:23.318 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.311+0000 7f9a12d95b80 1 --2- 172.21.15.23:0/671482033 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x55c87b3c28d0 0x55c87b3fe3c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:23.319 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.311+0000 7f9a12d95b80 1 --2- 172.21.15.23:0/671482033 >> [v2:172.21.15.23:6832/1046971212,v1:172.21.15.23:6833/1046971212] conn(0x55c87b460920 0x55c87b3fa7c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:23.319 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.311+0000 7f9a12d95b80 1 --2- 172.21.15.23:0/671482033 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c87b3e9d60 0x55c87b3e5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:23.319 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.311+0000 7f99d7fff700 0 client.4398 ms_handle_reset on v2:172.21.15.23:6832/1046971212 2023-03-27T10:02:23.319 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.311+0000 7f9a12d95b80 1 -- 172.21.15.23:0/671482033 >> 172.21.15.23:0/671482033 conn(0x55c87b3bda40 msgr2=0x55c87b3e8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:23.319 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.311+0000 7f9a12d95b80 1 -- 172.21.15.23:0/671482033 shutdown_connections 2023-03-27T10:02:23.319 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.311+0000 7f9a12d95b80 1 -- 172.21.15.23:0/671482033 wait complete. 2023-03-27T10:02:23.319 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.312+0000 7f9a12d95b80 20 client.4398 trim_cache size 0 max 16384 2023-03-27T10:02:23.320 INFO:teuthology.orchestra.run.smithi023.stdout:{ 2023-03-27T10:02:23.320 INFO:teuthology.orchestra.run.smithi023.stdout: "cluster_fsid": "1d8d6e6e-0d21-4f51-91ee-d9434d4fb750", 2023-03-27T10:02:23.320 INFO:teuthology.orchestra.run.smithi023.stdout: "whoami": 0, 2023-03-27T10:02:23.321 INFO:teuthology.orchestra.run.smithi023.stdout: "id": 4306, 2023-03-27T10:02:23.321 INFO:teuthology.orchestra.run.smithi023.stdout: "want_state": "up:active", 2023-03-27T10:02:23.321 INFO:teuthology.orchestra.run.smithi023.stdout: "state": "up:active", 2023-03-27T10:02:23.321 INFO:teuthology.orchestra.run.smithi023.stdout: "fs_name": "cephfs", 2023-03-27T10:02:23.321 INFO:teuthology.orchestra.run.smithi023.stdout: "rank_uptime": 10.983598014, 2023-03-27T10:02:23.321 INFO:teuthology.orchestra.run.smithi023.stdout: "mdsmap_epoch": 6, 2023-03-27T10:02:23.322 INFO:teuthology.orchestra.run.smithi023.stdout: "osdmap_epoch": 24, 2023-03-27T10:02:23.322 INFO:teuthology.orchestra.run.smithi023.stdout: "osdmap_epoch_barrier": 24, 2023-03-27T10:02:23.322 INFO:teuthology.orchestra.run.smithi023.stdout: "uptime": 16.445991876000001 2023-03-27T10:02:23.322 INFO:teuthology.orchestra.run.smithi023.stdout:} 2023-03-27T10:02:23.323 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.312+0000 7f99ff3ff700 1 -- 172.21.15.23:0/1319374964 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f99dc066e30 msgr2=0x7f99dc0692e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:23.323 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.312+0000 7f99ff3ff700 1 --2- 172.21.15.23:0/1319374964 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f99dc066e30 0x7f99dc0692e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f99f81568a0 tx=0x7f99e8007ed0 comp rx=0 tx=0).stop 2023-03-27T10:02:23.323 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.312+0000 7f99ff3ff700 1 -- 172.21.15.23:0/1319374964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99f806b300 msgr2=0x7f99f81518c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:23.323 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.312+0000 7f99ff3ff700 1 --2- 172.21.15.23:0/1319374964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99f806b300 0x7f99f81518c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f99f4009c40 tx=0x7f99f4004750 comp rx=0 tx=0).stop 2023-03-27T10:02:23.323 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.312+0000 7f99ff3ff700 1 -- 172.21.15.23:0/1319374964 shutdown_connections 2023-03-27T10:02:23.324 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.312+0000 7f99ff3ff700 1 --2- 172.21.15.23:0/1319374964 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f99f80c7320 0x7f99f81ee8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:23.324 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.312+0000 7f99ff3ff700 1 --2- 172.21.15.23:0/1319374964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99f806b300 0x7f99f81518c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:23.324 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.312+0000 7f99ff3ff700 1 --2- 172.21.15.23:0/1319374964 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f99dc066e30 0x7f99dc0692e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:23.324 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.312+0000 7f99ff3ff700 1 --2- 172.21.15.23:0/1319374964 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f99f80c6920 0x7f99f81ee3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:23.324 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.312+0000 7f99ff3ff700 1 -- 172.21.15.23:0/1319374964 >> 172.21.15.23:0/1319374964 conn(0x7f99f8065ca0 msgr2=0x7f99f80c8890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:23.324 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.313+0000 7f99ff3ff700 1 -- 172.21.15.23:0/1319374964 shutdown_connections 2023-03-27T10:02:23.325 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.313+0000 7f99ff3ff700 1 -- 172.21.15.23:0/1319374964 wait complete. 2023-03-27T10:02:23.331 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-03-27T10:02:23.332 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-03-27T10:02:23.333 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-03-27T10:02:23.525 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.525+0000 7f59113ff700 1 Processor -- start 2023-03-27T10:02:23.525 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.525+0000 7f59113ff700 1 -- start start 2023-03-27T10:02:23.526 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.527+0000 7f59113ff700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f590c0c7320 0x7f590c06f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:23.526 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.527+0000 7f59113ff700 1 -- --> v1:172.21.15.23:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f590c06e820 con 0x7f590c0c6920 2023-03-27T10:02:23.526 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.527+0000 7f59113ff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f590c0c86a0 con 0x7f590c06b300 2023-03-27T10:02:23.527 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.527+0000 7f59113ff700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f590c159690 con 0x7f590c0c7320 2023-03-27T10:02:23.527 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.527+0000 7f5903fff700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f590c0c7320 0x7f590c06f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:23.527 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.527+0000 7f5903fff700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f590c0c7320 0x7f590c06f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:34614/0 (socket says 172.21.15.23:34614) 2023-03-27T10:02:23.527 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.527+0000 7f590affd700 1 --1- >> v1:172.21.15.23:6789/0 conn(0x7f590c0c6920 0x7f590c0c7f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.23:6789/0 says I am v1:172.21.15.23:36736/0 (socket says 172.21.15.23:36736) 2023-03-27T10:02:23.527 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.527+0000 7f5903fff700 1 -- 172.21.15.23:0/2361985943 learned_addr learned my addr 172.21.15.23:0/2361985943 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:23.528 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.527+0000 7f5903fff700 1 -- 172.21.15.23:0/2361985943 >> v1:172.21.15.112:6789/0 conn(0x7f590c06b300 legacy=0x7f590c163150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:23.529 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.527+0000 7f5903fff700 1 -- 172.21.15.23:0/2361985943 >> v1:172.21.15.23:6789/0 conn(0x7f590c0c6920 legacy=0x7f590c0c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:23.529 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.527+0000 7f5903fff700 1 -- 172.21.15.23:0/2361985943 --> v2:172.21.15.23:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f590c06dc10 con 0x7f590c0c7320 2023-03-27T10:02:23.529 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.527+0000 7f590a7fc700 1 -- 172.21.15.23:0/2361985943 <== mon.0 v1:172.21.15.23:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 926819098 0 0) 0x7f590c06e820 con 0x7f590c0c6920 2023-03-27T10:02:23.529 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.528+0000 7f5903fff700 1 --2- 172.21.15.23:0/2361985943 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f590c0c7320 0x7f590c06f200 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f58f40098f0 tx=0x7f58f4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee132edea2b53c8 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:23.529 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.528+0000 7f590a7fc700 1 -- 172.21.15.23:0/2361985943 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f400b8a0 con 0x7f590c0c7320 2023-03-27T10:02:23.529 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.528+0000 7f590a7fc700 1 -- 172.21.15.23:0/2361985943 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f400ba00 con 0x7f590c0c7320 2023-03-27T10:02:23.530 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.528+0000 7f590a7fc700 1 -- 172.21.15.23:0/2361985943 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f400bd90 con 0x7f590c0c7320 2023-03-27T10:02:23.530 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.528+0000 7f59113ff700 1 -- 172.21.15.23:0/2361985943 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f590c0c7320 msgr2=0x7f590c06f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:23.530 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.528+0000 7f59113ff700 1 --2- 172.21.15.23:0/2361985943 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f590c0c7320 0x7f590c06f200 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f58f40098f0 tx=0x7f58f4009c00 comp rx=0 tx=0).stop 2023-03-27T10:02:23.530 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.529+0000 7f59113ff700 1 -- 172.21.15.23:0/2361985943 shutdown_connections 2023-03-27T10:02:23.530 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.529+0000 7f59113ff700 1 --2- 172.21.15.23:0/2361985943 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f590c0c7320 0x7f590c06f200 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:23.530 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.529+0000 7f59113ff700 1 -- 172.21.15.23:0/2361985943 >> 172.21.15.23:0/2361985943 conn(0x7f590c065ca0 msgr2=0x7f590c066070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:23.530 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.529+0000 7f59113ff700 1 -- 172.21.15.23:0/2361985943 shutdown_connections 2023-03-27T10:02:23.531 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.529+0000 7f59113ff700 1 -- 172.21.15.23:0/2361985943 wait complete. 2023-03-27T10:02:23.532 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.530+0000 7f59113ff700 1 Processor -- start 2023-03-27T10:02:23.532 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.530+0000 7f59113ff700 1 -- start start 2023-03-27T10:02:23.533 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.531+0000 7f59113ff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f590c06b300 0x7f590c14f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:23.533 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.531+0000 7f59113ff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f590c0c6920 0x7f590c1f2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:23.533 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.531+0000 7f59113ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f590c0c7320 0x7f590c14c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:23.533 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.531+0000 7f59113ff700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590c14f830 con 0x7f590c0c6920 2023-03-27T10:02:23.533 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.531+0000 7f59113ff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590c15ca90 con 0x7f590c06b300 2023-03-27T10:02:23.533 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.531+0000 7f59113ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f590c1f2c50 con 0x7f590c0c7320 2023-03-27T10:02:23.533 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.532+0000 7f5903fff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f590c0c6920 0x7f590c1f2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:23.534 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.532+0000 7f5903fff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f590c0c6920 0x7f590c1f2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:34628/0 (socket says 172.21.15.23:34628) 2023-03-27T10:02:23.534 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.532+0000 7f5903fff700 1 -- 172.21.15.23:0/411065163 learned_addr learned my addr 172.21.15.23:0/411065163 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:23.534 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.532+0000 7f590affd700 1 --2- 172.21.15.23:0/411065163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f590c06b300 0x7f590c14f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:23.534 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.532+0000 7f590b7fe700 1 --2- 172.21.15.23:0/411065163 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f590c0c7320 0x7f590c14c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:23.534 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.532+0000 7f5903fff700 1 -- 172.21.15.23:0/411065163 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f590c0c7320 msgr2=0x7f590c14c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:23.534 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.532+0000 7f5903fff700 1 --2- 172.21.15.23:0/411065163 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f590c0c7320 0x7f590c14c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:23.535 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.532+0000 7f5903fff700 1 -- 172.21.15.23:0/411065163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f590c06b300 msgr2=0x7f590c14f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:23.535 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.532+0000 7f5903fff700 1 --2- 172.21.15.23:0/411065163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f590c06b300 0x7f590c14f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:23.535 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.532+0000 7f5903fff700 1 -- 172.21.15.23:0/411065163 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f4009550 con 0x7f590c0c6920 2023-03-27T10:02:23.535 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.532+0000 7f590b7fe700 1 --2- 172.21.15.23:0/411065163 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f590c0c7320 0x7f590c14c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-27T10:02:23.535 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.532+0000 7f5903fff700 1 --2- 172.21.15.23:0/411065163 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f590c0c6920 0x7f590c1f2710 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f590c06dbb0 tx=0x7f58f4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:23.535 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.533+0000 7f59037fe700 1 -- 172.21.15.23:0/411065163 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f4011970 con 0x7f590c0c6920 2023-03-27T10:02:23.536 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.533+0000 7f59037fe700 1 -- 172.21.15.23:0/411065163 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f401f070 con 0x7f590c0c6920 2023-03-27T10:02:23.536 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.533+0000 7f59037fe700 1 -- 172.21.15.23:0/411065163 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f4011cd0 con 0x7f590c0c6920 2023-03-27T10:02:23.536 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.533+0000 7f59113ff700 1 -- 172.21.15.23:0/411065163 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f590c06e1d0 con 0x7f590c0c6920 2023-03-27T10:02:23.536 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.533+0000 7f59113ff700 1 -- 172.21.15.23:0/411065163 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f590c06def0 con 0x7f590c0c6920 2023-03-27T10:02:23.537 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.534+0000 7f59113ff700 1 -- 172.21.15.23:0/411065163 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f590c06e4b0 con 0x7f590c0c6920 2023-03-27T10:02:23.537 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.535+0000 7f59037fe700 1 -- 172.21.15.23:0/411065163 <== mon.0 v2:172.21.15.23:3300/0 4 ==== mgrmap(e 6) v1 ==== 85113+0+0 (secure 0 0 0) 0x7f58f400f050 con 0x7f590c0c6920 2023-03-27T10:02:23.538 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.535+0000 7f59037fe700 1 --2- 172.21.15.23:0/411065163 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f58ec066e30 0x7f58ec0692e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:23.538 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.536+0000 7f59037fe700 1 -- 172.21.15.23:0/411065163 <== mon.0 v2:172.21.15.23:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f58f4021070 con 0x7f590c0c6920 2023-03-27T10:02:23.538 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.536+0000 7f590affd700 1 --2- 172.21.15.23:0/411065163 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f58ec066e30 0x7f58ec0692e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:23.538 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.536+0000 7f590affd700 1 --2- 172.21.15.23:0/411065163 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f58ec066e30 0x7f58ec0692e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f58fc007b10 tx=0x7f58fc007a60 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:23.539 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.540+0000 7f59037fe700 1 -- 172.21.15.23:0/411065163 <== mon.0 v2:172.21.15.23:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173280 (secure 0 0 0) 0x7f58f4052f00 con 0x7f590c0c6920 2023-03-27T10:02:23.786 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.786+0000 7f59113ff700 1 -- 172.21.15.23:0/411065163 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f590c067a00 con 0x7f590c0c6920 2023-03-27T10:02:23.786 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.786+0000 7f59037fe700 1 -- 172.21.15.23:0/411065163 <== mon.0 v2:172.21.15.23:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7f58f402b070 con 0x7f590c0c6920 2023-03-27T10:02:23.787 INFO:teuthology.orchestra.run.smithi023.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-03-27T10:02:23.788 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.789+0000 7f59113ff700 1 -- 172.21.15.23:0/411065163 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f58ec066e30 msgr2=0x7f58ec0692e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:23.788 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.789+0000 7f59113ff700 1 --2- 172.21.15.23:0/411065163 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f58ec066e30 0x7f58ec0692e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f58fc007b10 tx=0x7f58fc007a60 comp rx=0 tx=0).stop 2023-03-27T10:02:23.788 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.789+0000 7f59113ff700 1 -- 172.21.15.23:0/411065163 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f590c0c6920 msgr2=0x7f590c1f2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:23.789 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.789+0000 7f59113ff700 1 --2- 172.21.15.23:0/411065163 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f590c0c6920 0x7f590c1f2710 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f590c06dbb0 tx=0x7f58f4004750 comp rx=0 tx=0).stop 2023-03-27T10:02:23.789 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.789+0000 7f59113ff700 1 -- 172.21.15.23:0/411065163 shutdown_connections 2023-03-27T10:02:23.789 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.789+0000 7f59113ff700 1 --2- 172.21.15.23:0/411065163 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f58ec066e30 0x7f58ec0692e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:23.789 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.789+0000 7f59113ff700 1 --2- 172.21.15.23:0/411065163 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f590c0c7320 0x7f590c14c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:23.789 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.789+0000 7f59113ff700 1 --2- 172.21.15.23:0/411065163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f590c06b300 0x7f590c14f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:23.789 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.789+0000 7f59113ff700 1 --2- 172.21.15.23:0/411065163 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f590c0c6920 0x7f590c1f2710 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:23.789 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.789+0000 7f59113ff700 1 -- 172.21.15.23:0/411065163 >> 172.21.15.23:0/411065163 conn(0x7f590c065ca0 msgr2=0x7f590c0cbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:23.790 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.789+0000 7f59113ff700 1 -- 172.21.15.23:0/411065163 shutdown_connections 2023-03-27T10:02:23.790 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:23.789+0000 7f59113ff700 1 -- 172.21.15.23:0/411065163 wait complete. 2023-03-27T10:02:23.807 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-03-27T10:02:23.808 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-03-27T10:02:23.809 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi023.front.sepia.ceph.com 2023-03-27T10:02:23.809 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-03-27T10:02:23.810 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-03-27T10:02:23.810 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-03-27T10:02:23.810 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-03-27T10:02:23.811 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-03-27T10:02:23.811 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-03-27T10:02:23.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-27T10:02:23.812 DEBUG:teuthology.orchestra.run.smithi023:> ip addr 2023-03-27T10:02:23.850 INFO:teuthology.orchestra.run.smithi023.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-03-27T10:02:23.851 INFO:teuthology.orchestra.run.smithi023.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-03-27T10:02:23.851 INFO:teuthology.orchestra.run.smithi023.stdout: inet 127.0.0.1/8 scope host lo 2023-03-27T10:02:23.851 INFO:teuthology.orchestra.run.smithi023.stdout: valid_lft forever preferred_lft forever 2023-03-27T10:02:23.851 INFO:teuthology.orchestra.run.smithi023.stdout: inet6 ::1/128 scope host 2023-03-27T10:02:23.851 INFO:teuthology.orchestra.run.smithi023.stdout: valid_lft forever preferred_lft forever 2023-03-27T10:02:23.851 INFO:teuthology.orchestra.run.smithi023.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-03-27T10:02:23.852 INFO:teuthology.orchestra.run.smithi023.stdout: link/ether 0c:c4:7a:6b:fd:ae brd ff:ff:ff:ff:ff:ff 2023-03-27T10:02:23.852 INFO:teuthology.orchestra.run.smithi023.stdout: altname enp7s0f0 2023-03-27T10:02:23.852 INFO:teuthology.orchestra.run.smithi023.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-03-27T10:02:23.852 INFO:teuthology.orchestra.run.smithi023.stdout: link/ether 0c:c4:7a:6b:fd:af brd ff:ff:ff:ff:ff:ff 2023-03-27T10:02:23.852 INFO:teuthology.orchestra.run.smithi023.stdout: altname enp7s0f1 2023-03-27T10:02:23.852 INFO:teuthology.orchestra.run.smithi023.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-03-27T10:02:23.852 INFO:teuthology.orchestra.run.smithi023.stdout: link/ether 0c:c4:7a:59:ce:42 brd ff:ff:ff:ff:ff:ff 2023-03-27T10:02:23.853 INFO:teuthology.orchestra.run.smithi023.stdout: altname enp1s0f0 2023-03-27T10:02:23.853 INFO:teuthology.orchestra.run.smithi023.stdout: inet 172.21.15.23/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-03-27T10:02:23.853 INFO:teuthology.orchestra.run.smithi023.stdout: valid_lft 42372sec preferred_lft 42372sec 2023-03-27T10:02:23.853 INFO:teuthology.orchestra.run.smithi023.stdout: inet6 fe80::ec4:7aff:fe59:ce42/64 scope link 2023-03-27T10:02:23.853 INFO:teuthology.orchestra.run.smithi023.stdout: valid_lft forever preferred_lft forever 2023-03-27T10:02:23.853 INFO:teuthology.orchestra.run.smithi023.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-03-27T10:02:23.853 INFO:teuthology.orchestra.run.smithi023.stdout: link/ether 0c:c4:7a:59:ce:43 brd ff:ff:ff:ff:ff:ff 2023-03-27T10:02:23.854 INFO:teuthology.orchestra.run.smithi023.stdout: altname enp1s0f1 2023-03-27T10:02:23.854 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-03-27T10:02:23.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-27T10:02:23.857 DEBUG:teuthology.orchestra.run.smithi023:> (cd / && exec bash -c ' 2023-03-27T10:02:23.857 DEBUG:teuthology.orchestra.run.smithi023:> set -e 2023-03-27T10:02:23.858 DEBUG:teuthology.orchestra.run.smithi023:> sudo ip link add name ceph-brx type bridge 2023-03-27T10:02:23.859 DEBUG:teuthology.orchestra.run.smithi023:> sudo ip addr flush dev ceph-brx 2023-03-27T10:02:23.859 DEBUG:teuthology.orchestra.run.smithi023:> sudo ip link set ceph-brx up 2023-03-27T10:02:23.860 DEBUG:teuthology.orchestra.run.smithi023:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-03-27T10:02:23.860 DEBUG:teuthology.orchestra.run.smithi023:> ') 2023-03-27T10:02:24.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-27T10:02:24.293 DEBUG:teuthology.orchestra.run.smithi023:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-03-27T10:02:24.357 INFO:teuthology.orchestra.run.smithi023.stdout:1 2023-03-27T10:02:24.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-27T10:02:24.360 DEBUG:teuthology.orchestra.run.smithi023:> route 2023-03-27T10:02:24.413 INFO:teuthology.orchestra.run.smithi023.stdout:Kernel IP routing table 2023-03-27T10:02:24.413 INFO:teuthology.orchestra.run.smithi023.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-03-27T10:02:24.414 INFO:teuthology.orchestra.run.smithi023.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-03-27T10:02:24.414 INFO:teuthology.orchestra.run.smithi023.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-03-27T10:02:24.414 INFO:teuthology.orchestra.run.smithi023.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-03-27T10:02:24.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-27T10:02:24.416 DEBUG:teuthology.orchestra.run.smithi023:> (cd / && exec bash -c ' 2023-03-27T10:02:24.416 DEBUG:teuthology.orchestra.run.smithi023:> set -e 2023-03-27T10:02:24.417 DEBUG:teuthology.orchestra.run.smithi023:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-03-27T10:02:24.418 DEBUG:teuthology.orchestra.run.smithi023:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-03-27T10:02:24.419 DEBUG:teuthology.orchestra.run.smithi023:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-03-27T10:02:24.419 DEBUG:teuthology.orchestra.run.smithi023:> ') 2023-03-27T10:02:25.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-27T10:02:25.079 DEBUG:teuthology.orchestra.run.smithi023:> ip netns list 2023-03-27T10:02:25.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-27T10:02:25.121 DEBUG:teuthology.orchestra.run.smithi023:> ip netns list-id 2023-03-27T10:02:25.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-27T10:02:25.199 DEBUG:teuthology.orchestra.run.smithi023:> (cd / && exec bash -c ' 2023-03-27T10:02:25.199 DEBUG:teuthology.orchestra.run.smithi023:> set -e 2023-03-27T10:02:25.200 DEBUG:teuthology.orchestra.run.smithi023:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-03-27T10:02:25.200 DEBUG:teuthology.orchestra.run.smithi023:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-03-27T10:02:25.200 DEBUG:teuthology.orchestra.run.smithi023:> ') 2023-03-27T10:02:25.365 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-03-27T10:02:25.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-27T10:02:25.366 DEBUG:teuthology.orchestra.run.smithi023:> (cd / && exec bash -c ' 2023-03-27T10:02:25.366 DEBUG:teuthology.orchestra.run.smithi023:> set -e 2023-03-27T10:02:25.367 DEBUG:teuthology.orchestra.run.smithi023:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-03-27T10:02:25.367 DEBUG:teuthology.orchestra.run.smithi023:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-03-27T10:02:25.367 DEBUG:teuthology.orchestra.run.smithi023:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-03-27T10:02:25.368 DEBUG:teuthology.orchestra.run.smithi023:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-03-27T10:02:25.368 DEBUG:teuthology.orchestra.run.smithi023:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-03-27T10:02:25.368 DEBUG:teuthology.orchestra.run.smithi023:> ') 2023-03-27T10:02:25.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-27T10:02:25.612 DEBUG:teuthology.orchestra.run.smithi023:> (cd / && exec bash -c ' 2023-03-27T10:02:25.613 DEBUG:teuthology.orchestra.run.smithi023:> set -e 2023-03-27T10:02:25.613 DEBUG:teuthology.orchestra.run.smithi023:> sudo ip link set brx.0 up 2023-03-27T10:02:25.614 DEBUG:teuthology.orchestra.run.smithi023:> sudo ip link set dev brx.0 master ceph-brx 2023-03-27T10:02:25.614 DEBUG:teuthology.orchestra.run.smithi023:> ') 2023-03-27T10:02:25.779 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-03-27T10:02:25.780 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-03-27T10:02:25.780 DEBUG:teuthology.orchestra.run.smithi023:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-03-27T10:02:25.819 INFO:teuthology.orchestra.run.smithi023.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-03-27T10:02:25.819 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-03-27T10:02:25.820 DEBUG:teuthology.orchestra.run.smithi023:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-03-27T10:02:25.897 DEBUG:teuthology.orchestra.run.smithi023:> sudo modprobe fuse 2023-03-27T10:02:25.999 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-03-27T10:02:26.000 DEBUG:teuthology.orchestra.run.smithi023:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-03-27T10:02:26.061 INFO:teuthology.orchestra.run.smithi023.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-03-27T10:02:26.064 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-03-27T10:02:26.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-27T10:02:26.066 DEBUG:teuthology.orchestra.run.smithi023:> ls /sys/fs/fuse/connections 2023-03-27T10:02:26.104 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-03-27T10:02:26.104 DEBUG:teuthology.orchestra.run.smithi023:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-03-27T10:02:26.148 DEBUG:teuthology.orchestra.run.smithi023:> sudo modprobe fuse 2023-03-27T10:02:26.212 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-03-27T10:02:26.213 DEBUG:teuthology.orchestra.run.smithi023:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-03-27T10:02:26.275 INFO:teuthology.orchestra.run.smithi023.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-03-27T10:02:26.277 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-03-27T10:02:26.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-27T10:02:26.278 DEBUG:teuthology.orchestra.run.smithi023:> ls /sys/fs/fuse/connections 2023-03-27T10:02:26.327 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi023.stderr:2023-03-27T10:02:26.327+0000 7f76aae325c0 -1 init, newargv = 0x562ccbef97d0 newargc=15 2023-03-27T10:02:26.327 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi023.stderr:ceph-fuse[102295]: starting ceph client 2023-03-27T10:02:26.335 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi023.stderr:ceph-fuse[102295]: starting fuse 2023-03-27T10:02:27.419 DEBUG:teuthology.orchestra.run.smithi023:> sudo modprobe fuse 2023-03-27T10:02:27.490 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-03-27T10:02:27.490 DEBUG:teuthology.orchestra.run.smithi023:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-03-27T10:02:27.553 INFO:teuthology.orchestra.run.smithi023.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-03-27T10:02:27.555 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-03-27T10:02:27.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-27T10:02:27.557 DEBUG:teuthology.orchestra.run.smithi023:> ls /sys/fs/fuse/connections 2023-03-27T10:02:27.596 INFO:teuthology.orchestra.run.smithi023.stdout:42 2023-03-27T10:02:27.597 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2023-03-27T10:02:27.598 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-03-27T10:02:27.598 DEBUG:teuthology.orchestra.run.smithi023:> import glob 2023-03-27T10:02:27.598 DEBUG:teuthology.orchestra.run.smithi023:> import re 2023-03-27T10:02:27.599 DEBUG:teuthology.orchestra.run.smithi023:> import os 2023-03-27T10:02:27.599 DEBUG:teuthology.orchestra.run.smithi023:> import subprocess 2023-03-27T10:02:27.599 DEBUG:teuthology.orchestra.run.smithi023:> 2023-03-27T10:02:27.599 DEBUG:teuthology.orchestra.run.smithi023:> def _find_admin_socket(client_name): 2023-03-27T10:02:27.600 DEBUG:teuthology.orchestra.run.smithi023:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-03-27T10:02:27.600 DEBUG:teuthology.orchestra.run.smithi023:> files = glob.glob(asok_path) 2023-03-27T10:02:27.600 DEBUG:teuthology.orchestra.run.smithi023:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-03-27T10:02:27.601 DEBUG:teuthology.orchestra.run.smithi023:> 2023-03-27T10:02:27.601 DEBUG:teuthology.orchestra.run.smithi023:> # Given a non-glob path, it better be there 2023-03-27T10:02:27.601 DEBUG:teuthology.orchestra.run.smithi023:> if "*" not in asok_path: 2023-03-27T10:02:27.602 DEBUG:teuthology.orchestra.run.smithi023:> assert(len(files) == 1) 2023-03-27T10:02:27.602 DEBUG:teuthology.orchestra.run.smithi023:> return files[0] 2023-03-27T10:02:27.602 DEBUG:teuthology.orchestra.run.smithi023:> 2023-03-27T10:02:27.602 DEBUG:teuthology.orchestra.run.smithi023:> for f in files: 2023-03-27T10:02:27.603 DEBUG:teuthology.orchestra.run.smithi023:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-03-27T10:02:27.603 DEBUG:teuthology.orchestra.run.smithi023:> if os.path.exists("/proc/{0}".format(pid)): 2023-03-27T10:02:27.603 DEBUG:teuthology.orchestra.run.smithi023:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-03-27T10:02:27.603 DEBUG:teuthology.orchestra.run.smithi023:> contents = proc_f.read() 2023-03-27T10:02:27.604 DEBUG:teuthology.orchestra.run.smithi023:> if mountpoint in contents: 2023-03-27T10:02:27.604 DEBUG:teuthology.orchestra.run.smithi023:> return f 2023-03-27T10:02:27.604 DEBUG:teuthology.orchestra.run.smithi023:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-03-27T10:02:27.604 DEBUG:teuthology.orchestra.run.smithi023:> 2023-03-27T10:02:27.605 DEBUG:teuthology.orchestra.run.smithi023:> print(_find_admin_socket("client.admin")) 2023-03-27T10:02:27.605 DEBUG:teuthology.orchestra.run.smithi023:> ' 2023-03-27T10:02:27.772 INFO:teuthology.orchestra.run.smithi023.stdout:/var/run/ceph/ceph-client.admin.102295.asok 2023-03-27T10:02:27.830 INFO:tasks.ceph.mds.a.smithi023.stderr:2023-03-27T10:02:27.830+0000 7f62b27f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-03-27T10:02:27.949 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.102295.asok 2023-03-27T10:02:27.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-27T10:02:27.950 DEBUG:teuthology.orchestra.run.smithi023:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.102295.asok status 2023-03-27T10:02:28.170 INFO:teuthology.orchestra.run.smithi023.stdout:{ 2023-03-27T10:02:28.171 INFO:teuthology.orchestra.run.smithi023.stdout: "metadata": { 2023-03-27T10:02:28.171 INFO:teuthology.orchestra.run.smithi023.stdout: "ceph_sha1": "b70fdf973c2ffa23c0d8aa6918519da05d922302", 2023-03-27T10:02:28.171 INFO:teuthology.orchestra.run.smithi023.stdout: "ceph_version": "ceph version 18.0.0-2533-gb70fdf97 (b70fdf973c2ffa23c0d8aa6918519da05d922302) reef (dev)", 2023-03-27T10:02:28.171 INFO:teuthology.orchestra.run.smithi023.stdout: "entity_id": "admin", 2023-03-27T10:02:28.172 INFO:teuthology.orchestra.run.smithi023.stdout: "hostname": "smithi023", 2023-03-27T10:02:28.172 INFO:teuthology.orchestra.run.smithi023.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-03-27T10:02:28.172 INFO:teuthology.orchestra.run.smithi023.stdout: "pid": "102295", 2023-03-27T10:02:28.172 INFO:teuthology.orchestra.run.smithi023.stdout: "root": "/" 2023-03-27T10:02:28.173 INFO:teuthology.orchestra.run.smithi023.stdout: }, 2023-03-27T10:02:28.173 INFO:teuthology.orchestra.run.smithi023.stdout: "dentry_count": 0, 2023-03-27T10:02:28.174 INFO:teuthology.orchestra.run.smithi023.stdout: "dentry_pinned_count": 0, 2023-03-27T10:02:28.174 INFO:teuthology.orchestra.run.smithi023.stdout: "id": 4413, 2023-03-27T10:02:28.174 INFO:teuthology.orchestra.run.smithi023.stdout: "inst": { 2023-03-27T10:02:28.174 INFO:teuthology.orchestra.run.smithi023.stdout: "name": { 2023-03-27T10:02:28.174 INFO:teuthology.orchestra.run.smithi023.stdout: "type": "client", 2023-03-27T10:02:28.174 INFO:teuthology.orchestra.run.smithi023.stdout: "num": 4413 2023-03-27T10:02:28.175 INFO:teuthology.orchestra.run.smithi023.stdout: }, 2023-03-27T10:02:28.175 INFO:teuthology.orchestra.run.smithi023.stdout: "addr": { 2023-03-27T10:02:28.175 INFO:teuthology.orchestra.run.smithi023.stdout: "type": "v1", 2023-03-27T10:02:28.175 INFO:teuthology.orchestra.run.smithi023.stdout: "addr": "192.168.0.1:0", 2023-03-27T10:02:28.175 INFO:teuthology.orchestra.run.smithi023.stdout: "nonce": 513401372 2023-03-27T10:02:28.176 INFO:teuthology.orchestra.run.smithi023.stdout: } 2023-03-27T10:02:28.176 INFO:teuthology.orchestra.run.smithi023.stdout: }, 2023-03-27T10:02:28.176 INFO:teuthology.orchestra.run.smithi023.stdout: "addr": { 2023-03-27T10:02:28.176 INFO:teuthology.orchestra.run.smithi023.stdout: "type": "v1", 2023-03-27T10:02:28.176 INFO:teuthology.orchestra.run.smithi023.stdout: "addr": "192.168.0.1:0", 2023-03-27T10:02:28.176 INFO:teuthology.orchestra.run.smithi023.stdout: "nonce": 513401372 2023-03-27T10:02:28.176 INFO:teuthology.orchestra.run.smithi023.stdout: }, 2023-03-27T10:02:28.177 INFO:teuthology.orchestra.run.smithi023.stdout: "inst_str": "client.4413 192.168.0.1:0/513401372", 2023-03-27T10:02:28.177 INFO:teuthology.orchestra.run.smithi023.stdout: "addr_str": "192.168.0.1:0/513401372", 2023-03-27T10:02:28.177 INFO:teuthology.orchestra.run.smithi023.stdout: "inode_count": 1, 2023-03-27T10:02:28.177 INFO:teuthology.orchestra.run.smithi023.stdout: "mds_epoch": 6, 2023-03-27T10:02:28.177 INFO:teuthology.orchestra.run.smithi023.stdout: "osd_epoch": 24, 2023-03-27T10:02:28.177 INFO:teuthology.orchestra.run.smithi023.stdout: "osd_epoch_barrier": 0, 2023-03-27T10:02:28.178 INFO:teuthology.orchestra.run.smithi023.stdout: "blocklisted": false, 2023-03-27T10:02:28.178 INFO:teuthology.orchestra.run.smithi023.stdout: "fs_name": "cephfs" 2023-03-27T10:02:28.178 INFO:teuthology.orchestra.run.smithi023.stdout:} 2023-03-27T10:02:28.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-27T10:02:28.184 DEBUG:teuthology.orchestra.run.smithi023:> stat --file-system '--printf=%T 2023-03-27T10:02:28.184 DEBUG:teuthology.orchestra.run.smithi023:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-03-27T10:02:28.223 INFO:teuthology.orchestra.run.smithi023.stdout:fuseblk 2023-03-27T10:02:28.224 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-03-27T10:02:28.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-27T10:02:28.225 DEBUG:teuthology.orchestra.run.smithi023:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-03-27T10:02:28.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-27T10:02:28.328 DEBUG:teuthology.orchestra.run.smithi023:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-03-27T10:02:28.780 INFO:teuthology.orchestra.run.smithi023.stdout:# file: . 2023-03-27T10:02:28.781 INFO:teuthology.orchestra.run.smithi023.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-03-27T10:02:28.781 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:02:28.782 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2023-03-27T10:02:28.782 DEBUG:teuthology.orchestra.run.smithi023:> dd if=/proc/self/mounts of=/dev/stdout 2023-03-27T10:02:28.823 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-27T10:02:29.055 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.056+0000 7fe996fff700 1 Processor -- start 2023-03-27T10:02:29.056 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.056+0000 7fe996fff700 1 -- start start 2023-03-27T10:02:29.056 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.057+0000 7fe996fff700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7fe9900c9520 0x7fe9900c8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:29.057 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.057+0000 7fe996fff700 1 -- --> v1:172.21.15.23:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9900ca730 con 0x7fe9900c6990 2023-03-27T10:02:29.057 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.057+0000 7fe996fff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9900cf9f0 con 0x7fe99006b440 2023-03-27T10:02:29.057 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.057+0000 7fe996fff700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe990158a00 con 0x7fe9900c9520 2023-03-27T10:02:29.058 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.057+0000 7fe994d9b700 1 --1- >> v1:172.21.15.23:6789/0 conn(0x7fe9900c6990 0x7fe9900c6d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.23:6789/0 says I am v1:172.21.15.23:59254/0 (socket says 172.21.15.23:59254) 2023-03-27T10:02:29.059 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.057+0000 7fe994d9b700 1 -- 172.21.15.23:0/1741345608 learned_addr learned my addr 172.21.15.23:0/1741345608 (peer_addr_for_me v1:172.21.15.23:0/0) 2023-03-27T10:02:29.059 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.057+0000 7fe98ffff700 1 --2- 172.21.15.23:0/1741345608 >> v2:172.21.15.23:3300/0 conn(0x7fe9900c9520 0x7fe9900c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:29.059 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.058+0000 7fe98ffff700 1 -- 172.21.15.23:0/1741345608 >> v1:172.21.15.112:6789/0 conn(0x7fe99006b440 legacy=0x7fe99006b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:29.059 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.058+0000 7fe98ffff700 1 -- 172.21.15.23:0/1741345608 >> v1:172.21.15.23:6789/0 conn(0x7fe9900c6990 legacy=0x7fe9900c6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:29.059 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.058+0000 7fe98ffff700 1 -- 172.21.15.23:0/1741345608 --> v2:172.21.15.23:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe990158ce0 con 0x7fe9900c9520 2023-03-27T10:02:29.059 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.058+0000 7fe98f7fe700 1 -- 172.21.15.23:0/1741345608 <== mon.0 v1:172.21.15.23:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2198344698 0 0) 0x7fe9900ca730 con 0x7fe9900c6990 2023-03-27T10:02:29.059 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.058+0000 7fe98ffff700 1 --2- 172.21.15.23:0/1741345608 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fe9900c9520 0x7fe9900c8820 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe97c009840 tx=0x7fe97c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f96888e8966c385 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:29.060 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.058+0000 7fe98f7fe700 1 -- 172.21.15.23:0/1741345608 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97c011830 con 0x7fe9900c9520 2023-03-27T10:02:29.060 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.058+0000 7fe98f7fe700 1 -- 172.21.15.23:0/1741345608 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe97c011990 con 0x7fe9900c9520 2023-03-27T10:02:29.060 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.058+0000 7fe98f7fe700 1 -- 172.21.15.23:0/1741345608 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97c011d20 con 0x7fe9900c9520 2023-03-27T10:02:29.060 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.059+0000 7fe996fff700 1 -- 172.21.15.23:0/1741345608 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fe9900c9520 msgr2=0x7fe9900c8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:29.060 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.059+0000 7fe996fff700 1 --2- 172.21.15.23:0/1741345608 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fe9900c9520 0x7fe9900c8820 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe97c009840 tx=0x7fe97c009b50 comp rx=0 tx=0).stop 2023-03-27T10:02:29.060 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.059+0000 7fe996fff700 1 -- 172.21.15.23:0/1741345608 shutdown_connections 2023-03-27T10:02:29.060 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.059+0000 7fe996fff700 1 --2- 172.21.15.23:0/1741345608 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fe9900c9520 0x7fe9900c8820 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:29.060 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.059+0000 7fe996fff700 1 -- 172.21.15.23:0/1741345608 >> 172.21.15.23:0/1741345608 conn(0x7fe990071920 msgr2=0x7fe990071cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:29.061 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.059+0000 7fe996fff700 1 -- 172.21.15.23:0/1741345608 shutdown_connections 2023-03-27T10:02:29.061 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.059+0000 7fe996fff700 1 -- 172.21.15.23:0/1741345608 wait complete. 2023-03-27T10:02:29.063 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.061+0000 7fe996fff700 1 Processor -- start 2023-03-27T10:02:29.063 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.061+0000 7fe996fff700 1 -- start start 2023-03-27T10:02:29.063 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.062+0000 7fe996fff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe99006b440 0x7fe990154660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:29.063 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.062+0000 7fe996fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9900c6990 0x7fe9901ee810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:29.064 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.062+0000 7fe996fff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fe9900c9520 0x7fe9901eed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:29.064 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.062+0000 7fe996fff700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9901ee3e0 con 0x7fe9900c9520 2023-03-27T10:02:29.064 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.062+0000 7fe996fff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9901e3000 con 0x7fe9900c6990 2023-03-27T10:02:29.064 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.062+0000 7fe996fff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9901559b0 con 0x7fe99006b440 2023-03-27T10:02:29.064 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.062+0000 7fe994d9b700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe99006b440 0x7fe990154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:29.064 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.062+0000 7fe98ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9900c6990 0x7fe9901ee810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:29.064 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.062+0000 7fe99559c700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fe9900c9520 0x7fe9901eed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:29.065 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.062+0000 7fe994d9b700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe99006b440 0x7fe990154660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.23:60066/0 (socket says 172.21.15.23:60066) 2023-03-27T10:02:29.065 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.062+0000 7fe994d9b700 1 -- 172.21.15.23:0/923720137 learned_addr learned my addr 172.21.15.23:0/923720137 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:29.065 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.063+0000 7fe99559c700 1 -- 172.21.15.23:0/923720137 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe99006b440 msgr2=0x7fe990154660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:29.065 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.063+0000 7fe99559c700 1 --2- 172.21.15.23:0/923720137 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe99006b440 0x7fe990154660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:29.065 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.063+0000 7fe99559c700 1 -- 172.21.15.23:0/923720137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9900c6990 msgr2=0x7fe9901ee810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:29.065 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.063+0000 7fe99559c700 1 --2- 172.21.15.23:0/923720137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9900c6990 0x7fe9901ee810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:29.065 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.063+0000 7fe99559c700 1 -- 172.21.15.23:0/923720137 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe97c0094a0 con 0x7fe9900c9520 2023-03-27T10:02:29.066 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.063+0000 7fe99559c700 1 --2- 172.21.15.23:0/923720137 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fe9900c9520 0x7fe9901eed50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe98800cb90 tx=0x7fe98800cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:29.066 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.063+0000 7fe98d7fa700 1 -- 172.21.15.23:0/923720137 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe988003c00 con 0x7fe9900c9520 2023-03-27T10:02:29.066 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.063+0000 7fe98d7fa700 1 -- 172.21.15.23:0/923720137 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe988003d60 con 0x7fe9900c9520 2023-03-27T10:02:29.066 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.063+0000 7fe98d7fa700 1 -- 172.21.15.23:0/923720137 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98800aa80 con 0x7fe9900c9520 2023-03-27T10:02:29.066 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.063+0000 7fe996fff700 1 -- 172.21.15.23:0/923720137 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9901592a0 con 0x7fe9900c9520 2023-03-27T10:02:29.066 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.063+0000 7fe996fff700 1 -- 172.21.15.23:0/923720137 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe990158fc0 con 0x7fe9900c9520 2023-03-27T10:02:29.067 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.064+0000 7fe996fff700 1 -- 172.21.15.23:0/923720137 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe990159580 con 0x7fe9900c9520 2023-03-27T10:02:29.067 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.065+0000 7fe98d7fa700 1 -- 172.21.15.23:0/923720137 <== mon.0 v2:172.21.15.23:3300/0 4 ==== mgrmap(e 6) v1 ==== 85113+0+0 (secure 0 0 0) 0x7fe988010050 con 0x7fe9900c9520 2023-03-27T10:02:29.067 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.066+0000 7fe98d7fa700 1 --2- 172.21.15.23:0/923720137 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fe974066e60 0x7fe974069310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:29.068 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.067+0000 7fe98d7fa700 1 -- 172.21.15.23:0/923720137 <== mon.0 v2:172.21.15.23:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe988084380 con 0x7fe9900c9520 2023-03-27T10:02:29.068 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.067+0000 7fe994d9b700 1 --2- 172.21.15.23:0/923720137 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fe974066e60 0x7fe974069310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:29.068 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.068+0000 7fe994d9b700 1 --2- 172.21.15.23:0/923720137 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fe974066e60 0x7fe974069310 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe984001340 tx=0x7fe9840012d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:29.069 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.070+0000 7fe98d7fa700 1 -- 172.21.15.23:0/923720137 <== mon.0 v2:172.21.15.23:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173280 (secure 0 0 0) 0x7fe9880512d0 con 0x7fe9900c9520 2023-03-27T10:02:29.269 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.269+0000 7fe996fff700 1 -- 172.21.15.23:0/923720137 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9901ee530 con 0x7fe9900c9520 2023-03-27T10:02:29.270 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.271+0000 7fe98d7fa700 1 -- 172.21.15.23:0/923720137 <== mon.0 v2:172.21.15.23:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+19798 (secure 0 0 0) 0x7fe9901ee530 con 0x7fe9900c9520 2023-03-27T10:02:29.271 INFO:teuthology.orchestra.run.smithi023.stdout: 2023-03-27T10:02:29.271 INFO:teuthology.orchestra.run.smithi023.stdout:{"epoch":24,"fsid":"1d8d6e6e-0d21-4f51-91ee-d9434d4fb750","created":"2023-03-27T10:01:53.805778+0000","modified":"2023-03-27T10:02:17.274631+0000","last_up_change":"2023-03-27T10:02:04.172266+0000","last_in_change":"2023-03-27T10:01:56.475091+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-27T10:02:05.288883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-27T10:02:05.643442+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-27T10:02:10.720339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-27T10:02:11.254847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-27T10:02:14.809537+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"df086052-536b-4fb1-bcce-7aaf5a13911e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6800","nonce":96642},{"type":"v1","addr":"172.21.15.23:6801","nonce":96642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6802","nonce":96642},{"type":"v1","addr":"172.21.15.23:6803","nonce":96642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6806","nonce":96642},{"type":"v1","addr":"172.21.15.23:6807","nonce":96642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6804","nonce":96642},{"type":"v1","addr":"172.21.15.23:6805","nonce":96642}]},"public_addr":"172.21.15.23:6801/96642","cluster_addr":"172.21.15.23:6803/96642","heartbeat_back_addr":"172.21.15.23:6807/96642","heartbeat_front_addr":"172.21.15.23:6805/96642","state":["exists","up"]},{"osd":1,"uuid":"03138001-a3fd-43fc-9797-a47643d9a3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6808","nonce":96643},{"type":"v1","addr":"172.21.15.23:6809","nonce":96643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6810","nonce":96643},{"type":"v1","addr":"172.21.15.23:6811","nonce":96643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6814","nonce":96643},{"type":"v1","addr":"172.21.15.23:6815","nonce":96643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6812","nonce":96643},{"type":"v1","addr":"172.21.15.23:6813","nonce":96643}]},"public_addr":"172.21.15.23:6809/96643","cluster_addr":"172.21.15.23:6811/96643","heartbeat_back_addr":"172.21.15.23:6815/96643","heartbeat_front_addr":"172.21.15.23:6813/96643","state":["exists","up"]},{"osd":2,"uuid":"964dd926-1160-44ef-aa34-b18965dc05f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6817","nonce":96644},{"type":"v1","addr":"172.21.15.23:6819","nonce":96644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6820","nonce":96644},{"type":"v1","addr":"172.21.15.23:6822","nonce":96644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6827","nonce":96644},{"type":"v1","addr":"172.21.15.23:6829","nonce":96644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6824","nonce":96644},{"type":"v1","addr":"172.21.15.23:6825","nonce":96644}]},"public_addr":"172.21.15.23:6819/96644","cluster_addr":"172.21.15.23:6822/96644","heartbeat_back_addr":"172.21.15.23:6829/96644","heartbeat_front_addr":"172.21.15.23:6825/96644","state":["exists","up"]},{"osd":3,"uuid":"4ae5500f-cb24-44a8-a536-5c399c09e566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6816","nonce":96645},{"type":"v1","addr":"172.21.15.23:6818","nonce":96645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6821","nonce":96645},{"type":"v1","addr":"172.21.15.23:6823","nonce":96645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6830","nonce":96645},{"type":"v1","addr":"172.21.15.23:6831","nonce":96645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.23:6826","nonce":96645},{"type":"v1","addr":"172.21.15.23:6828","nonce":96645}]},"public_addr":"172.21.15.23:6818/96645","cluster_addr":"172.21.15.23:6823/96645","heartbeat_back_addr":"172.21.15.23:6831/96645","heartbeat_front_addr":"172.21.15.23:6828/96645","state":["exists","up"]},{"osd":4,"uuid":"de228ab2-92b4-40bc-8dbd-8f2f4895dc22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":127986},{"type":"v1","addr":"172.21.15.112:6809","nonce":127986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":127986},{"type":"v1","addr":"172.21.15.112:6811","nonce":127986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6815","nonce":127986},{"type":"v1","addr":"172.21.15.112:6818","nonce":127986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":127986},{"type":"v1","addr":"172.21.15.112:6813","nonce":127986}]},"public_addr":"172.21.15.112:6809/127986","cluster_addr":"172.21.15.112:6811/127986","heartbeat_back_addr":"172.21.15.112:6818/127986","heartbeat_front_addr":"172.21.15.112:6813/127986","state":["exists","up"]},{"osd":5,"uuid":"6c839571-b490-4e66-92ac-0d7301c2e677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":127987},{"type":"v1","addr":"172.21.15.112:6801","nonce":127987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":127987},{"type":"v1","addr":"172.21.15.112:6803","nonce":127987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":127987},{"type":"v1","addr":"172.21.15.112:6807","nonce":127987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":127987},{"type":"v1","addr":"172.21.15.112:6805","nonce":127987}]},"public_addr":"172.21.15.112:6801/127987","cluster_addr":"172.21.15.112:6803/127987","heartbeat_back_addr":"172.21.15.112:6807/127987","heartbeat_front_addr":"172.21.15.112:6805/127987","state":["exists","up"]},{"osd":6,"uuid":"871b1a2b-f0ed-4faf-8f9a-a5a0973e11c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":127988},{"type":"v1","addr":"172.21.15.112:6816","nonce":127988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":127988},{"type":"v1","addr":"172.21.15.112:6819","nonce":127988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":127988},{"type":"v1","addr":"172.21.15.112:6823","nonce":127988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":127988},{"type":"v1","addr":"172.21.15.112:6821","nonce":127988}]},"public_addr":"172.21.15.112:6816/127988","cluster_addr":"172.21.15.112:6819/127988","heartbeat_back_addr":"172.21.15.112:6823/127988","heartbeat_front_addr":"172.21.15.112:6821/127988","state":["exists","up"]},{"osd":7,"uuid":"d8228353-34b5-4363-84d8-642512f5bc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":128009},{"type":"v1","addr":"172.21.15.112:6825","nonce":128009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":128009},{"type":"v1","addr":"172.21.15.112:6827","nonce":128009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":128009},{"type":"v1","addr":"172.21.15.112:6831","nonce":128009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":128009},{"type":"v1","addr":"172.21.15.112:6829","nonce":128009}]},"public_addr":"172.21.15.112:6825/128009","cluster_addr":"172.21.15.112:6827/128009","heartbeat_back_addr":"172.21.15.112:6831/128009","heartbeat_front_addr":"172.21.15.112:6829/128009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:02.132160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:01.882517+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:01.927673+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:01.867525+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:01.723580+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:01.730394+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:01.716687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-27T10:02:01.744322+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-27T10:02:29.272 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.273+0000 7fe996fff700 1 -- 172.21.15.23:0/923720137 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fe974066e60 msgr2=0x7fe974069310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:29.273 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.273+0000 7fe996fff700 1 --2- 172.21.15.23:0/923720137 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fe974066e60 0x7fe974069310 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe984001340 tx=0x7fe9840012d0 comp rx=0 tx=0).stop 2023-03-27T10:02:29.273 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.273+0000 7fe996fff700 1 -- 172.21.15.23:0/923720137 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fe9900c9520 msgr2=0x7fe9901eed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:29.273 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.273+0000 7fe996fff700 1 --2- 172.21.15.23:0/923720137 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fe9900c9520 0x7fe9901eed50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe98800cb90 tx=0x7fe98800cea0 comp rx=0 tx=0).stop 2023-03-27T10:02:29.273 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.273+0000 7fe996fff700 1 -- 172.21.15.23:0/923720137 shutdown_connections 2023-03-27T10:02:29.274 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.273+0000 7fe996fff700 1 --2- 172.21.15.23:0/923720137 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fe9900c9520 0x7fe9901eed50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:29.274 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.273+0000 7fe996fff700 1 --2- 172.21.15.23:0/923720137 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fe974066e60 0x7fe974069310 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:29.274 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.273+0000 7fe996fff700 1 --2- 172.21.15.23:0/923720137 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe99006b440 0x7fe990154660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:29.274 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.273+0000 7fe996fff700 1 --2- 172.21.15.23:0/923720137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9900c6990 0x7fe9901ee810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:29.274 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.273+0000 7fe996fff700 1 -- 172.21.15.23:0/923720137 >> 172.21.15.23:0/923720137 conn(0x7fe990071920 msgr2=0x7fe9900c87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:29.274 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.273+0000 7fe996fff700 1 -- 172.21.15.23:0/923720137 shutdown_connections 2023-03-27T10:02:29.274 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.273+0000 7fe996fff700 1 -- 172.21.15.23:0/923720137 wait complete. 2023-03-27T10:02:29.291 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-03-27T10:02:05.288883+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-03-27T10:02:05.643442+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-03-27T10:02:10.720339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-03-27T10:02:11.254847+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-03-27T10:02:14.809537+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-03-27T10:02:29.292 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-27T10:02:29.485 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.485+0000 7fd5f9bff700 1 Processor -- start 2023-03-27T10:02:29.485 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.486+0000 7fd5f9bff700 1 -- start start 2023-03-27T10:02:29.487 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.487+0000 7fd5f9bff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5f40c6990 0x7fd5f40c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:29.487 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.487+0000 7fd5f9bff700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7fd5f406b440 0x7fd5f406b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:29.487 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.488+0000 7fd5f9bff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5f40ca730 con 0x7fd5f40c9520 2023-03-27T10:02:29.488 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.488+0000 7fd5f9bff700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5f40c72a0 con 0x7fd5f406b440 2023-03-27T10:02:29.488 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.488+0000 7fd5f9bff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f40cf9f0 con 0x7fd5f40c6990 2023-03-27T10:02:29.488 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.488+0000 7fd5f3fff700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7fd5f406b440 0x7fd5f406b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:29.488 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.488+0000 7fd5f3fff700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7fd5f406b440 0x7fd5f406b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:42420/0 (socket says 172.21.15.23:42420) 2023-03-27T10:02:29.488 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.488+0000 7fd5f37fe700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5f40c6990 0x7fd5f40c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:29.488 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.488+0000 7fd5f3fff700 1 -- 172.21.15.23:0/536495898 learned_addr learned my addr 172.21.15.23:0/536495898 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:29.488 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.488+0000 7fd5ebfff700 1 --1- 172.21.15.23:0/536495898 >> v1:172.21.15.112:6789/0 conn(0x7fd5f40c9520 0x7fd5f40c8820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.23:34452/0 (socket says 172.21.15.23:34452) 2023-03-27T10:02:29.489 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.488+0000 7fd5f3fff700 1 -- 172.21.15.23:0/536495898 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5f40c6990 msgr2=0x7fd5f40c6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:02:29.490 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.488+0000 7fd5f3fff700 1 --2- 172.21.15.23:0/536495898 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5f40c6990 0x7fd5f40c6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:29.490 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.488+0000 7fd5f3fff700 1 -- 172.21.15.23:0/536495898 >> v1:172.21.15.112:6789/0 conn(0x7fd5f40c9520 legacy=0x7fd5f40c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:29.490 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.488+0000 7fd5f3fff700 1 -- 172.21.15.23:0/536495898 --> v2:172.21.15.23:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f40c8f30 con 0x7fd5f406b440 2023-03-27T10:02:29.490 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.488+0000 7fd5f2ffd700 1 -- 172.21.15.23:0/536495898 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1600529231 0 0) 0x7fd5f40ca730 con 0x7fd5f40c9520 2023-03-27T10:02:29.490 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.488+0000 7fd5f3fff700 1 --2- 172.21.15.23:0/536495898 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fd5f406b440 0x7fd5f406b860 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec009e50 tx=0x7fd5ec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16af074d9f551dd2 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:29.490 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.489+0000 7fd5f2ffd700 1 -- 172.21.15.23:0/536495898 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec00e070 con 0x7fd5f406b440 2023-03-27T10:02:29.491 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.489+0000 7fd5f2ffd700 1 -- 172.21.15.23:0/536495898 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5ec00b910 con 0x7fd5f406b440 2023-03-27T10:02:29.491 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.489+0000 7fd5f2ffd700 1 -- 172.21.15.23:0/536495898 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec00bcc0 con 0x7fd5f406b440 2023-03-27T10:02:29.491 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.489+0000 7fd5f9bff700 1 -- 172.21.15.23:0/536495898 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fd5f406b440 msgr2=0x7fd5f406b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:29.491 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.489+0000 7fd5f9bff700 1 --2- 172.21.15.23:0/536495898 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fd5f406b440 0x7fd5f406b860 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec009e50 tx=0x7fd5ec00b330 comp rx=0 tx=0).stop 2023-03-27T10:02:29.491 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.489+0000 7fd5f9bff700 1 -- 172.21.15.23:0/536495898 shutdown_connections 2023-03-27T10:02:29.491 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.489+0000 7fd5f9bff700 1 --2- 172.21.15.23:0/536495898 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fd5f406b440 0x7fd5f406b860 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:29.491 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.489+0000 7fd5f9bff700 1 --2- 172.21.15.23:0/536495898 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5f40c6990 0x7fd5f40c6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:29.491 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.489+0000 7fd5f9bff700 1 -- 172.21.15.23:0/536495898 >> 172.21.15.23:0/536495898 conn(0x7fd5f4071920 msgr2=0x7fd5f4071cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:29.492 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.489+0000 7fd5f9bff700 1 -- 172.21.15.23:0/536495898 shutdown_connections 2023-03-27T10:02:29.492 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.489+0000 7fd5f9bff700 1 -- 172.21.15.23:0/536495898 wait complete. 2023-03-27T10:02:29.493 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.491+0000 7fd5f9bff700 1 Processor -- start 2023-03-27T10:02:29.493 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.491+0000 7fd5f9bff700 1 -- start start 2023-03-27T10:02:29.493 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.492+0000 7fd5f9bff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5f406b440 0x7fd5f41f2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:29.494 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.492+0000 7fd5f9bff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fd5f40c6990 0x7fd5f4153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:29.494 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.492+0000 7fd5f9bff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5f40c9520 0x7fd5f4065f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:29.494 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.492+0000 7fd5f9bff700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f41dfa30 con 0x7fd5f40c6990 2023-03-27T10:02:29.494 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.492+0000 7fd5f9bff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f41f2ec0 con 0x7fd5f406b440 2023-03-27T10:02:29.494 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.492+0000 7fd5f9bff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f41decb0 con 0x7fd5f40c9520 2023-03-27T10:02:29.494 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.492+0000 7fd5ebfff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fd5f40c6990 0x7fd5f4153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:29.494 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.492+0000 7fd5f37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5f406b440 0x7fd5f41f2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:29.495 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.492+0000 7fd5f3fff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5f40c9520 0x7fd5f4065f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:29.495 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.492+0000 7fd5f37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5f406b440 0x7fd5f41f2980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.23:53580/0 (socket says 172.21.15.23:53580) 2023-03-27T10:02:29.495 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.492+0000 7fd5f37fe700 1 -- 172.21.15.23:0/662607070 learned_addr learned my addr 172.21.15.23:0/662607070 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:29.495 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.493+0000 7fd5f3fff700 1 -- 172.21.15.23:0/662607070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5f406b440 msgr2=0x7fd5f41f2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:29.495 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.493+0000 7fd5f3fff700 1 --2- 172.21.15.23:0/662607070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5f406b440 0x7fd5f41f2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:29.495 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.493+0000 7fd5f3fff700 1 -- 172.21.15.23:0/662607070 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fd5f40c6990 msgr2=0x7fd5f4153980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:29.495 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.493+0000 7fd5f3fff700 1 --2- 172.21.15.23:0/662607070 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fd5f40c6990 0x7fd5f4153980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:29.495 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.493+0000 7fd5f3fff700 1 -- 172.21.15.23:0/662607070 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5ec009b30 con 0x7fd5f40c9520 2023-03-27T10:02:29.496 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.493+0000 7fd5ebfff700 1 --2- 172.21.15.23:0/662607070 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fd5f40c6990 0x7fd5f4153980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-27T10:02:29.496 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.493+0000 7fd5f3fff700 1 --2- 172.21.15.23:0/662607070 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5f40c9520 0x7fd5f4065f10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec015000 tx=0x7fd5ec004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:29.496 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.493+0000 7fd5f0ff9700 1 -- 172.21.15.23:0/662607070 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec004880 con 0x7fd5f40c9520 2023-03-27T10:02:29.496 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.493+0000 7fd5f0ff9700 1 -- 172.21.15.23:0/662607070 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5ec0099b0 con 0x7fd5f40c9520 2023-03-27T10:02:29.496 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.493+0000 7fd5f9bff700 1 -- 172.21.15.23:0/662607070 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5f4158ce0 con 0x7fd5f40c9520 2023-03-27T10:02:29.496 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.493+0000 7fd5f0ff9700 1 -- 172.21.15.23:0/662607070 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec01b670 con 0x7fd5f40c9520 2023-03-27T10:02:29.496 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.494+0000 7fd5f9bff700 1 -- 172.21.15.23:0/662607070 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5f4158a00 con 0x7fd5f40c9520 2023-03-27T10:02:29.497 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.494+0000 7fd5f9bff700 1 -- 172.21.15.23:0/662607070 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5f4158fc0 con 0x7fd5f40c9520 2023-03-27T10:02:29.497 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.496+0000 7fd5f0ff9700 1 -- 172.21.15.23:0/662607070 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 6) v1 ==== 85113+0+0 (secure 0 0 0) 0x7fd5ec021030 con 0x7fd5f40c9520 2023-03-27T10:02:29.498 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.496+0000 7fd5f0ff9700 1 --2- 172.21.15.23:0/662607070 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fd5d4066e30 0x7fd5d40692e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:29.498 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.497+0000 7fd5f0ff9700 1 -- 172.21.15.23:0/662607070 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd5ec085d50 con 0x7fd5f40c9520 2023-03-27T10:02:29.498 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.497+0000 7fd5f37fe700 1 --2- 172.21.15.23:0/662607070 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fd5d4066e30 0x7fd5d40692e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:29.498 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.497+0000 7fd5f37fe700 1 --2- 172.21.15.23:0/662607070 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fd5d4066e30 0x7fd5d40692e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4009ca0 tx=0x7fd5e4009210 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:29.499 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.500+0000 7fd5f0ff9700 1 -- 172.21.15.23:0/662607070 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173280 (secure 0 0 0) 0x7fd5ec052c40 con 0x7fd5f40c9520 2023-03-27T10:02:29.699 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.700+0000 7fd5f9bff700 1 -- 172.21.15.23:0/662607070 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd5f4074320 con 0x7fd5f40c9520 2023-03-27T10:02:29.700 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.701+0000 7fd5f0ff9700 1 -- 172.21.15.23:0/662607070 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd5ec056260 con 0x7fd5f40c9520 2023-03-27T10:02:29.700 INFO:teuthology.orchestra.run.smithi023.stdout:pg_num: 1 2023-03-27T10:02:29.703 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.703+0000 7fd5f9bff700 1 -- 172.21.15.23:0/662607070 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fd5d4066e30 msgr2=0x7fd5d40692e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:29.703 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.703+0000 7fd5f9bff700 1 --2- 172.21.15.23:0/662607070 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fd5d4066e30 0x7fd5d40692e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4009ca0 tx=0x7fd5e4009210 comp rx=0 tx=0).stop 2023-03-27T10:02:29.703 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.703+0000 7fd5f9bff700 1 -- 172.21.15.23:0/662607070 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5f40c9520 msgr2=0x7fd5f4065f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:29.703 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.703+0000 7fd5f9bff700 1 --2- 172.21.15.23:0/662607070 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5f40c9520 0x7fd5f4065f10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec015000 tx=0x7fd5ec004ad0 comp rx=0 tx=0).stop 2023-03-27T10:02:29.703 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.703+0000 7fd5f9bff700 1 -- 172.21.15.23:0/662607070 shutdown_connections 2023-03-27T10:02:29.703 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.703+0000 7fd5f9bff700 1 --2- 172.21.15.23:0/662607070 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fd5d4066e30 0x7fd5d40692e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:29.704 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.703+0000 7fd5f9bff700 1 --2- 172.21.15.23:0/662607070 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5f40c9520 0x7fd5f4065f10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:29.704 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.703+0000 7fd5f9bff700 1 --2- 172.21.15.23:0/662607070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5f406b440 0x7fd5f41f2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:29.704 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.703+0000 7fd5f9bff700 1 --2- 172.21.15.23:0/662607070 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fd5f40c6990 0x7fd5f4153980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:29.704 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.703+0000 7fd5f9bff700 1 -- 172.21.15.23:0/662607070 >> 172.21.15.23:0/662607070 conn(0x7fd5f4071920 msgr2=0x7fd5f40cdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:29.704 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.703+0000 7fd5f9bff700 1 -- 172.21.15.23:0/662607070 shutdown_connections 2023-03-27T10:02:29.704 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.703+0000 7fd5f9bff700 1 -- 172.21.15.23:0/662607070 wait complete. 2023-03-27T10:02:29.720 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-27T10:02:29.914 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.914+0000 7fb87d5ff700 1 Processor -- start 2023-03-27T10:02:29.914 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.914+0000 7fb87d5ff700 1 -- start start 2023-03-27T10:02:29.915 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.915+0000 7fb87d5ff700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7fb8780c6920 0x7fb87806f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:29.915 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.916+0000 7fb87d5ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb87806b300 0x7fb87806b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:29.915 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.916+0000 7fb87d5ff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb87806e820 con 0x7fb8780c7320 2023-03-27T10:02:29.915 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.916+0000 7fb87d5ff700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8780c86a0 con 0x7fb8780c6920 2023-03-27T10:02:29.916 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.916+0000 7fb87d5ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb878159690 con 0x7fb87806b300 2023-03-27T10:02:29.916 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.916+0000 7fb876ffd700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7fb8780c6920 0x7fb87806f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:29.916 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.916+0000 7fb8777fe700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb87806b300 0x7fb87806b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:29.916 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.916+0000 7fb86ffff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fb8780c7320 0x7fb8780c7f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.23:34462/0 (socket says 172.21.15.23:34462) 2023-03-27T10:02:29.916 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.916+0000 7fb86ffff700 1 -- 172.21.15.23:0/2767662194 learned_addr learned my addr 172.21.15.23:0/2767662194 (peer_addr_for_me v1:172.21.15.23:0/0) 2023-03-27T10:02:29.918 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.916+0000 7fb876ffd700 1 -- 172.21.15.23:0/2767662194 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb87806b300 msgr2=0x7fb87806b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:02:29.918 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.916+0000 7fb8767fc700 1 -- 172.21.15.23:0/2767662194 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 784087231 0 0) 0x7fb87806e820 con 0x7fb8780c7320 2023-03-27T10:02:29.918 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.916+0000 7fb876ffd700 1 --2- 172.21.15.23:0/2767662194 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb87806b300 0x7fb87806b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:29.918 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.916+0000 7fb876ffd700 1 -- 172.21.15.23:0/2767662194 >> v1:172.21.15.112:6789/0 conn(0x7fb8780c7320 legacy=0x7fb8780c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:29.918 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.916+0000 7fb876ffd700 1 -- 172.21.15.23:0/2767662194 --> v2:172.21.15.23:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb87806dc10 con 0x7fb8780c6920 2023-03-27T10:02:29.918 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.916+0000 7fb8777fe700 1 --2- 172.21.15.23:0/2767662194 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb87806b300 0x7fb87806b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-27T10:02:29.919 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.917+0000 7fb876ffd700 1 --2- 172.21.15.23:0/2767662194 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fb8780c6920 0x7fb87806f200 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb868009840 tx=0x7fb868009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cab6668fef5c199 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:29.919 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.917+0000 7fb8767fc700 1 -- 172.21.15.23:0/2767662194 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb868011830 con 0x7fb8780c6920 2023-03-27T10:02:29.919 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.917+0000 7fb8767fc700 1 -- 172.21.15.23:0/2767662194 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb868011990 con 0x7fb8780c6920 2023-03-27T10:02:29.919 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.917+0000 7fb8767fc700 1 -- 172.21.15.23:0/2767662194 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb868011d20 con 0x7fb8780c6920 2023-03-27T10:02:29.919 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.917+0000 7fb87d5ff700 1 -- 172.21.15.23:0/2767662194 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fb8780c6920 msgr2=0x7fb87806f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:29.919 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.917+0000 7fb87d5ff700 1 --2- 172.21.15.23:0/2767662194 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fb8780c6920 0x7fb87806f200 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb868009840 tx=0x7fb868009b50 comp rx=0 tx=0).stop 2023-03-27T10:02:29.919 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.918+0000 7fb87d5ff700 1 -- 172.21.15.23:0/2767662194 shutdown_connections 2023-03-27T10:02:29.920 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.918+0000 7fb87d5ff700 1 --2- 172.21.15.23:0/2767662194 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fb8780c6920 0x7fb87806f200 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:29.920 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.918+0000 7fb87d5ff700 1 --2- 172.21.15.23:0/2767662194 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb87806b300 0x7fb87806b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:29.920 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.918+0000 7fb87d5ff700 1 -- 172.21.15.23:0/2767662194 >> 172.21.15.23:0/2767662194 conn(0x7fb878065ca0 msgr2=0x7fb878066070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:29.920 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.918+0000 7fb87d5ff700 1 -- 172.21.15.23:0/2767662194 shutdown_connections 2023-03-27T10:02:29.920 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.918+0000 7fb87d5ff700 1 -- 172.21.15.23:0/2767662194 wait complete. 2023-03-27T10:02:29.922 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.919+0000 7fb87d5ff700 1 Processor -- start 2023-03-27T10:02:29.922 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.920+0000 7fb87d5ff700 1 -- start start 2023-03-27T10:02:29.922 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.920+0000 7fb87d5ff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb87806b300 0x7fb878155a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:29.922 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.920+0000 7fb87d5ff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fb8780c6920 0x7fb8780adbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:29.922 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.921+0000 7fb87d5ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb8780c7320 0x7fb8780ae110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:29.922 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.921+0000 7fb87d5ff700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8780af6d0 con 0x7fb8780c6920 2023-03-27T10:02:29.922 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.921+0000 7fb87d5ff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8780ac8e0 con 0x7fb87806b300 2023-03-27T10:02:29.923 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.921+0000 7fb87d5ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8780acb30 con 0x7fb8780c7320 2023-03-27T10:02:29.923 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.921+0000 7fb86ffff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fb8780c6920 0x7fb8780adbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:29.923 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.921+0000 7fb86ffff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fb8780c6920 0x7fb8780adbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:42446/0 (socket says 172.21.15.23:42446) 2023-03-27T10:02:29.923 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.921+0000 7fb876ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb87806b300 0x7fb878155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:29.923 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.921+0000 7fb86ffff700 1 -- 172.21.15.23:0/1625900014 learned_addr learned my addr 172.21.15.23:0/1625900014 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:29.923 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.921+0000 7fb8777fe700 1 --2- 172.21.15.23:0/1625900014 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb8780c7320 0x7fb8780ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:29.923 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.921+0000 7fb86ffff700 1 -- 172.21.15.23:0/1625900014 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb8780c7320 msgr2=0x7fb8780ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:29.923 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.921+0000 7fb86ffff700 1 --2- 172.21.15.23:0/1625900014 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb8780c7320 0x7fb8780ae110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:29.924 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.921+0000 7fb86ffff700 1 -- 172.21.15.23:0/1625900014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb87806b300 msgr2=0x7fb878155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:29.924 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.921+0000 7fb86ffff700 1 --2- 172.21.15.23:0/1625900014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb87806b300 0x7fb878155a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:29.924 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.921+0000 7fb86ffff700 1 -- 172.21.15.23:0/1625900014 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb870009a70 con 0x7fb8780c6920 2023-03-27T10:02:29.924 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.921+0000 7fb86ffff700 1 --2- 172.21.15.23:0/1625900014 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fb8780c6920 0x7fb8780adbd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb86000c550 tx=0x7fb86000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:29.924 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.922+0000 7fb86f7fe700 1 -- 172.21.15.23:0/1625900014 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb860004c80 con 0x7fb8780c6920 2023-03-27T10:02:29.924 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.922+0000 7fb86f7fe700 1 -- 172.21.15.23:0/1625900014 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb860004e00 con 0x7fb8780c6920 2023-03-27T10:02:29.924 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.922+0000 7fb86f7fe700 1 -- 172.21.15.23:0/1625900014 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb860007310 con 0x7fb8780c6920 2023-03-27T10:02:29.925 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.922+0000 7fb87d5ff700 1 -- 172.21.15.23:0/1625900014 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8680094a0 con 0x7fb8780c6920 2023-03-27T10:02:29.925 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.922+0000 7fb87d5ff700 1 -- 172.21.15.23:0/1625900014 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb87806e1d0 con 0x7fb8780c6920 2023-03-27T10:02:29.925 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.923+0000 7fb87d5ff700 1 -- 172.21.15.23:0/1625900014 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb87806def0 con 0x7fb8780c6920 2023-03-27T10:02:29.926 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.924+0000 7fb86f7fe700 1 -- 172.21.15.23:0/1625900014 <== mon.0 v2:172.21.15.23:3300/0 4 ==== mgrmap(e 6) v1 ==== 85113+0+0 (secure 0 0 0) 0x7fb86001a020 con 0x7fb8780c6920 2023-03-27T10:02:29.926 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.925+0000 7fb86f7fe700 1 --2- 172.21.15.23:0/1625900014 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fb858066e30 0x7fb8580692e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:29.926 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.925+0000 7fb86f7fe700 1 -- 172.21.15.23:0/1625900014 <== mon.0 v2:172.21.15.23:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb860084080 con 0x7fb8780c6920 2023-03-27T10:02:29.926 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.925+0000 7fb876ffd700 1 --2- 172.21.15.23:0/1625900014 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fb858066e30 0x7fb8580692e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:29.926 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.926+0000 7fb876ffd700 1 --2- 172.21.15.23:0/1625900014 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fb858066e30 0x7fb8580692e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb8680090b0 tx=0x7fb868008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:29.929 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:29.930+0000 7fb86f7fe700 1 -- 172.21.15.23:0/1625900014 <== mon.0 v2:172.21.15.23:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173280 (secure 0 0 0) 0x7fb860050f70 con 0x7fb8780c6920 2023-03-27T10:02:30.127 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.128+0000 7fb87d5ff700 1 -- 172.21.15.23:0/1625900014 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb87806e4b0 con 0x7fb8780c6920 2023-03-27T10:02:30.128 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.128+0000 7fb86f7fe700 1 -- 172.21.15.23:0/1625900014 <== mon.0 v2:172.21.15.23:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb860054590 con 0x7fb8780c6920 2023-03-27T10:02:30.128 INFO:teuthology.orchestra.run.smithi023.stdout:pg_num: 8 2023-03-27T10:02:30.130 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.130+0000 7fb87d5ff700 1 -- 172.21.15.23:0/1625900014 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fb858066e30 msgr2=0x7fb8580692e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:30.130 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.130+0000 7fb87d5ff700 1 --2- 172.21.15.23:0/1625900014 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fb858066e30 0x7fb8580692e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb8680090b0 tx=0x7fb868008030 comp rx=0 tx=0).stop 2023-03-27T10:02:30.130 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.130+0000 7fb87d5ff700 1 -- 172.21.15.23:0/1625900014 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fb8780c6920 msgr2=0x7fb8780adbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:30.131 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.130+0000 7fb87d5ff700 1 --2- 172.21.15.23:0/1625900014 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fb8780c6920 0x7fb8780adbd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb86000c550 tx=0x7fb86000c860 comp rx=0 tx=0).stop 2023-03-27T10:02:30.131 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.131+0000 7fb87d5ff700 1 -- 172.21.15.23:0/1625900014 shutdown_connections 2023-03-27T10:02:30.131 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.131+0000 7fb87d5ff700 1 --2- 172.21.15.23:0/1625900014 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fb858066e30 0x7fb8580692e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:30.131 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.131+0000 7fb87d5ff700 1 --2- 172.21.15.23:0/1625900014 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb8780c7320 0x7fb8780ae110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:30.131 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.131+0000 7fb87d5ff700 1 --2- 172.21.15.23:0/1625900014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb87806b300 0x7fb878155a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:30.132 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.131+0000 7fb87d5ff700 1 --2- 172.21.15.23:0/1625900014 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fb8780c6920 0x7fb8780adbd0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:30.132 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.131+0000 7fb87d5ff700 1 -- 172.21.15.23:0/1625900014 >> 172.21.15.23:0/1625900014 conn(0x7fb878065ca0 msgr2=0x7fb878066070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:30.133 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.131+0000 7fb87d5ff700 1 -- 172.21.15.23:0/1625900014 shutdown_connections 2023-03-27T10:02:30.133 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.131+0000 7fb87d5ff700 1 -- 172.21.15.23:0/1625900014 wait complete. 2023-03-27T10:02:30.147 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-03-27T10:02:30.339 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.340+0000 7fba467ff700 1 Processor -- start 2023-03-27T10:02:30.340 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.340+0000 7fba467ff700 1 -- start start 2023-03-27T10:02:30.342 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.341+0000 7fba467ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba400c6990 0x7fba400c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:30.342 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.341+0000 7fba467ff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fba400c9520 0x7fba400c8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:30.342 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.341+0000 7fba467ff700 1 -- --> v1:172.21.15.23:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba400ca730 con 0x7fba4006b440 2023-03-27T10:02:30.342 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.341+0000 7fba467ff700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba400c72a0 con 0x7fba400c9520 2023-03-27T10:02:30.343 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.341+0000 7fba467ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba400cf9f0 con 0x7fba400c6990 2023-03-27T10:02:30.343 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.342+0000 7fba3ffff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba400c6990 0x7fba400c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:30.343 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.342+0000 7fba44d9c700 1 --1- >> v1:172.21.15.23:6789/0 conn(0x7fba4006b440 0x7fba4006b860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.23:6789/0 says I am v1:172.21.15.23:59268/0 (socket says 172.21.15.23:59268) 2023-03-27T10:02:30.343 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.342+0000 7fba44d9c700 1 -- 172.21.15.23:0/1169777759 learned_addr learned my addr 172.21.15.23:0/1169777759 (peer_addr_for_me v1:172.21.15.23:0/0) 2023-03-27T10:02:30.343 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.342+0000 7fba3f7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fba400c9520 0x7fba400c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:30.343 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.342+0000 7fba3effd700 1 -- 172.21.15.23:0/1169777759 <== mon.0 v1:172.21.15.23:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4136291496 0 0) 0x7fba400ca730 con 0x7fba4006b440 2023-03-27T10:02:30.343 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.342+0000 7fba3ffff700 1 -- 172.21.15.23:0/1169777759 >> v2:172.21.15.112:3300/0 conn(0x7fba400c9520 msgr2=0x7fba400c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:02:30.344 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.342+0000 7fba3ffff700 1 --2- 172.21.15.23:0/1169777759 >> v2:172.21.15.112:3300/0 conn(0x7fba400c9520 0x7fba400c8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:30.344 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.342+0000 7fba3ffff700 1 -- 172.21.15.23:0/1169777759 >> v1:172.21.15.23:6789/0 conn(0x7fba4006b440 legacy=0x7fba4006b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:30.344 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.342+0000 7fba3ffff700 1 -- 172.21.15.23:0/1169777759 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba40158a00 con 0x7fba400c6990 2023-03-27T10:02:30.344 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.342+0000 7fba3ffff700 1 --2- 172.21.15.23:0/1169777759 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba400c6990 0x7fba400c6d60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fba30009840 tx=0x7fba30009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aba5c87df3872bae server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:30.344 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.342+0000 7fba3effd700 1 -- 172.21.15.23:0/1169777759 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba30011830 con 0x7fba400c6990 2023-03-27T10:02:30.344 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.342+0000 7fba3effd700 1 -- 172.21.15.23:0/1169777759 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba30011990 con 0x7fba400c6990 2023-03-27T10:02:30.344 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.343+0000 7fba3effd700 1 -- 172.21.15.23:0/1169777759 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba30011d20 con 0x7fba400c6990 2023-03-27T10:02:30.347 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.343+0000 7fba467ff700 1 -- 172.21.15.23:0/1169777759 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba400c6990 msgr2=0x7fba400c6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:30.347 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.343+0000 7fba467ff700 1 --2- 172.21.15.23:0/1169777759 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba400c6990 0x7fba400c6d60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fba30009840 tx=0x7fba30009b50 comp rx=0 tx=0).stop 2023-03-27T10:02:30.347 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.343+0000 7fba467ff700 1 -- 172.21.15.23:0/1169777759 shutdown_connections 2023-03-27T10:02:30.347 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.343+0000 7fba467ff700 1 --2- 172.21.15.23:0/1169777759 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba400c6990 0x7fba400c6d60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:30.347 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.343+0000 7fba467ff700 1 --2- 172.21.15.23:0/1169777759 >> v2:172.21.15.112:3300/0 conn(0x7fba400c9520 0x7fba400c8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:30.348 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.343+0000 7fba467ff700 1 -- 172.21.15.23:0/1169777759 >> 172.21.15.23:0/1169777759 conn(0x7fba40071920 msgr2=0x7fba40071cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:30.348 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.343+0000 7fba467ff700 1 -- 172.21.15.23:0/1169777759 shutdown_connections 2023-03-27T10:02:30.348 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.343+0000 7fba467ff700 1 -- 172.21.15.23:0/1169777759 wait complete. 2023-03-27T10:02:30.348 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.345+0000 7fba467ff700 1 Processor -- start 2023-03-27T10:02:30.348 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.345+0000 7fba467ff700 1 -- start start 2023-03-27T10:02:30.348 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.346+0000 7fba467ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba4006b440 0x7fba400cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:30.349 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.346+0000 7fba467ff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fba400c6990 0x7fba401ef270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:30.349 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.346+0000 7fba467ff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba400c9520 0x7fba401ef7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:30.349 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.346+0000 7fba467ff700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba4015a060 con 0x7fba400c6990 2023-03-27T10:02:30.349 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.346+0000 7fba467ff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba400ce410 con 0x7fba400c9520 2023-03-27T10:02:30.349 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.346+0000 7fba467ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba4014de00 con 0x7fba4006b440 2023-03-27T10:02:30.350 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.346+0000 7fba3f7fe700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fba400c6990 0x7fba401ef270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:30.350 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.346+0000 7fba3f7fe700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fba400c6990 0x7fba401ef270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:42448/0 (socket says 172.21.15.23:42448) 2023-03-27T10:02:30.350 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.346+0000 7fba44d9c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba400c9520 0x7fba401ef7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:30.350 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.346+0000 7fba3ffff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba4006b440 0x7fba400cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:30.351 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.346+0000 7fba3f7fe700 1 -- 172.21.15.23:0/2092869461 learned_addr learned my addr 172.21.15.23:0/2092869461 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:30.351 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.347+0000 7fba3f7fe700 1 -- 172.21.15.23:0/2092869461 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba4006b440 msgr2=0x7fba400cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:30.351 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.347+0000 7fba3f7fe700 1 --2- 172.21.15.23:0/2092869461 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba4006b440 0x7fba400cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:30.351 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.347+0000 7fba3f7fe700 1 -- 172.21.15.23:0/2092869461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba400c9520 msgr2=0x7fba401ef7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:30.352 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.347+0000 7fba3f7fe700 1 --2- 172.21.15.23:0/2092869461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba400c9520 0x7fba401ef7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:30.352 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.347+0000 7fba3f7fe700 1 -- 172.21.15.23:0/2092869461 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba300094a0 con 0x7fba400c6990 2023-03-27T10:02:30.352 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.347+0000 7fba3f7fe700 1 --2- 172.21.15.23:0/2092869461 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fba400c6990 0x7fba401ef270 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fba3400e870 tx=0x7fba3400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:30.352 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.347+0000 7fba3cff9700 1 -- 172.21.15.23:0/2092869461 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba34009590 con 0x7fba400c6990 2023-03-27T10:02:30.352 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.347+0000 7fba3cff9700 1 -- 172.21.15.23:0/2092869461 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba34013680 con 0x7fba400c6990 2023-03-27T10:02:30.352 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.347+0000 7fba467ff700 1 -- 172.21.15.23:0/2092869461 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba40158fc0 con 0x7fba400c6990 2023-03-27T10:02:30.353 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.347+0000 7fba3cff9700 1 -- 172.21.15.23:0/2092869461 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba340139f0 con 0x7fba400c6990 2023-03-27T10:02:30.353 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.347+0000 7fba467ff700 1 -- 172.21.15.23:0/2092869461 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba40158ce0 con 0x7fba400c6990 2023-03-27T10:02:30.354 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.348+0000 7fba467ff700 1 -- 172.21.15.23:0/2092869461 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba401592a0 con 0x7fba400c6990 2023-03-27T10:02:30.354 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.349+0000 7fba3cff9700 1 -- 172.21.15.23:0/2092869461 <== mon.0 v2:172.21.15.23:3300/0 4 ==== mgrmap(e 6) v1 ==== 85113+0+0 (secure 0 0 0) 0x7fba34023070 con 0x7fba400c6990 2023-03-27T10:02:30.354 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.350+0000 7fba3cff9700 1 --2- 172.21.15.23:0/2092869461 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fba2c066e30 0x7fba2c0692e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:30.354 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.350+0000 7fba3cff9700 1 -- 172.21.15.23:0/2092869461 <== mon.0 v2:172.21.15.23:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fba34084500 con 0x7fba400c6990 2023-03-27T10:02:30.354 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.350+0000 7fba3ffff700 1 --2- 172.21.15.23:0/2092869461 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fba2c066e30 0x7fba2c0692e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:30.355 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.351+0000 7fba3ffff700 1 --2- 172.21.15.23:0/2092869461 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fba2c066e30 0x7fba2c0692e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fba300090b0 tx=0x7fba30008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:30.358 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.354+0000 7fba3cff9700 1 -- 172.21.15.23:0/2092869461 <== mon.0 v2:172.21.15.23:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173280 (secure 0 0 0) 0x7fba340514e0 con 0x7fba400c6990 2023-03-27T10:02:30.554 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.555+0000 7fba467ff700 1 -- 172.21.15.23:0/2092869461 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fba400c66f0 con 0x7fba400c6990 2023-03-27T10:02:30.555 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.555+0000 7fba3cff9700 1 -- 172.21.15.23:0/2092869461 <== mon.0 v2:172.21.15.23:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fba34054b00 con 0x7fba400c6990 2023-03-27T10:02:30.555 INFO:teuthology.orchestra.run.smithi023.stdout:pg_num: 32 2023-03-27T10:02:30.557 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.557+0000 7fba467ff700 1 -- 172.21.15.23:0/2092869461 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fba2c066e30 msgr2=0x7fba2c0692e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:30.557 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.557+0000 7fba467ff700 1 --2- 172.21.15.23:0/2092869461 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fba2c066e30 0x7fba2c0692e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fba300090b0 tx=0x7fba30008030 comp rx=0 tx=0).stop 2023-03-27T10:02:30.557 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.557+0000 7fba467ff700 1 -- 172.21.15.23:0/2092869461 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fba400c6990 msgr2=0x7fba401ef270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:30.558 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.557+0000 7fba467ff700 1 --2- 172.21.15.23:0/2092869461 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fba400c6990 0x7fba401ef270 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fba3400e870 tx=0x7fba3400eb80 comp rx=0 tx=0).stop 2023-03-27T10:02:30.558 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.557+0000 7fba467ff700 1 -- 172.21.15.23:0/2092869461 shutdown_connections 2023-03-27T10:02:30.558 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.557+0000 7fba467ff700 1 --2- 172.21.15.23:0/2092869461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba400c9520 0x7fba401ef7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:30.558 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.557+0000 7fba467ff700 1 --2- 172.21.15.23:0/2092869461 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fba2c066e30 0x7fba2c0692e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:30.558 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.557+0000 7fba467ff700 1 --2- 172.21.15.23:0/2092869461 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba4006b440 0x7fba400cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:30.558 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.558+0000 7fba467ff700 1 --2- 172.21.15.23:0/2092869461 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fba400c6990 0x7fba401ef270 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:30.558 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.558+0000 7fba467ff700 1 -- 172.21.15.23:0/2092869461 >> 172.21.15.23:0/2092869461 conn(0x7fba40071920 msgr2=0x7fba400cca70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:30.559 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.558+0000 7fba467ff700 1 -- 172.21.15.23:0/2092869461 shutdown_connections 2023-03-27T10:02:30.559 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.558+0000 7fba467ff700 1 -- 172.21.15.23:0/2092869461 wait complete. 2023-03-27T10:02:30.574 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-03-27T10:02:30.765 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.765+0000 7f540a9ff700 1 Processor -- start 2023-03-27T10:02:30.765 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.766+0000 7f540a9ff700 1 -- start start 2023-03-27T10:02:30.766 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.767+0000 7f540a9ff700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f54040c6990 0x7f54040c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:30.767 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.767+0000 7f540a9ff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f540406b440 0x7f540406b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:30.767 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.767+0000 7f540a9ff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54040ca730 con 0x7f54040c9520 2023-03-27T10:02:30.767 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.767+0000 7f540a9ff700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54040c72a0 con 0x7f54040c6990 2023-03-27T10:02:30.767 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.767+0000 7f540a9ff700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54040cf9f0 con 0x7f540406b440 2023-03-27T10:02:30.767 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.767+0000 7f5403fff700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f54040c6990 0x7f54040c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:30.768 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.767+0000 7f5403fff700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f54040c6990 0x7f54040c6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:42454/0 (socket says 172.21.15.23:42454) 2023-03-27T10:02:30.768 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.767+0000 7f5403fff700 1 -- 172.21.15.23:0/1311006445 learned_addr learned my addr 172.21.15.23:0/1311006445 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:30.768 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.767+0000 7f5408f9c700 1 --2- 172.21.15.23:0/1311006445 >> v2:172.21.15.112:3300/0 conn(0x7f540406b440 0x7f540406b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:30.770 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.767+0000 7f5403fff700 1 -- 172.21.15.23:0/1311006445 >> v2:172.21.15.112:3300/0 conn(0x7f540406b440 msgr2=0x7f540406b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:02:30.771 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.768+0000 7f5403fff700 1 --2- 172.21.15.23:0/1311006445 >> v2:172.21.15.112:3300/0 conn(0x7f540406b440 0x7f540406b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:30.771 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.768+0000 7f5403fff700 1 -- 172.21.15.23:0/1311006445 >> v1:172.21.15.112:6789/0 conn(0x7f54040c9520 legacy=0x7f54040c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:30.772 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.768+0000 7f5403fff700 1 -- 172.21.15.23:0/1311006445 --> v2:172.21.15.23:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54040c8f30 con 0x7f54040c6990 2023-03-27T10:02:30.772 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.768+0000 7f5403fff700 1 --2- 172.21.15.23:0/1311006445 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f54040c6990 0x7f54040c6d60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f540406c7f0 tx=0x7f53f4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4de160b1d7a25ff0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:30.772 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.768+0000 7f5402ffd700 1 -- 172.21.15.23:0/1311006445 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f4011830 con 0x7f54040c6990 2023-03-27T10:02:30.773 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.768+0000 7f5402ffd700 1 -- 172.21.15.23:0/1311006445 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53f4011990 con 0x7f54040c6990 2023-03-27T10:02:30.773 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.768+0000 7f5402ffd700 1 -- 172.21.15.23:0/1311006445 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f4011d20 con 0x7f54040c6990 2023-03-27T10:02:30.773 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.769+0000 7f540a9ff700 1 -- 172.21.15.23:0/1311006445 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f54040c6990 msgr2=0x7f54040c6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:30.774 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.769+0000 7f540a9ff700 1 --2- 172.21.15.23:0/1311006445 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f54040c6990 0x7f54040c6d60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f540406c7f0 tx=0x7f53f4009bd0 comp rx=0 tx=0).stop 2023-03-27T10:02:30.774 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.769+0000 7f540a9ff700 1 -- 172.21.15.23:0/1311006445 shutdown_connections 2023-03-27T10:02:30.774 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.769+0000 7f540a9ff700 1 --2- 172.21.15.23:0/1311006445 >> v2:172.21.15.112:3300/0 conn(0x7f540406b440 0x7f540406b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:30.775 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.769+0000 7f540a9ff700 1 --2- 172.21.15.23:0/1311006445 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f54040c6990 0x7f54040c6d60 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:30.775 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.769+0000 7f540a9ff700 1 -- 172.21.15.23:0/1311006445 >> 172.21.15.23:0/1311006445 conn(0x7f5404071920 msgr2=0x7f5404071cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:30.776 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.769+0000 7f540a9ff700 1 -- 172.21.15.23:0/1311006445 shutdown_connections 2023-03-27T10:02:30.776 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.769+0000 7f540a9ff700 1 -- 172.21.15.23:0/1311006445 wait complete. 2023-03-27T10:02:30.776 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.771+0000 7f540a9ff700 1 Processor -- start 2023-03-27T10:02:30.776 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.771+0000 7f540a9ff700 1 -- start start 2023-03-27T10:02:30.781 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.772+0000 7f540a9ff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f540406b440 0x7f54041f2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:30.781 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.773+0000 7f540a9ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54040c6990 0x7f5404157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:30.781 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.773+0000 7f540a9ff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54040c9520 0x7f54041f02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:30.781 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.773+0000 7f540a9ff700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540414fb20 con 0x7f540406b440 2023-03-27T10:02:30.782 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.773+0000 7f540a9ff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54041efe50 con 0x7f54040c9520 2023-03-27T10:02:30.782 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.773+0000 7f540a9ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5404153580 con 0x7f54040c6990 2023-03-27T10:02:30.782 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.773+0000 7f5403fff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f540406b440 0x7f54041f2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:30.782 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.773+0000 7f5403fff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f540406b440 0x7f54041f2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:42464/0 (socket says 172.21.15.23:42464) 2023-03-27T10:02:30.782 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.773+0000 7f5403fff700 1 -- 172.21.15.23:0/3737390731 learned_addr learned my addr 172.21.15.23:0/3737390731 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:30.782 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.773+0000 7f5408f9c700 1 --2- 172.21.15.23:0/3737390731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54040c9520 0x7f54041f02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:30.782 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.773+0000 7f54037fe700 1 --2- 172.21.15.23:0/3737390731 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54040c6990 0x7f5404157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:30.783 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.773+0000 7f5403fff700 1 -- 172.21.15.23:0/3737390731 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54040c6990 msgr2=0x7f5404157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:30.783 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.773+0000 7f5403fff700 1 --2- 172.21.15.23:0/3737390731 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54040c6990 0x7f5404157f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:30.783 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.773+0000 7f5403fff700 1 -- 172.21.15.23:0/3737390731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54040c9520 msgr2=0x7f54041f02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:30.783 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.773+0000 7f5403fff700 1 --2- 172.21.15.23:0/3737390731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54040c9520 0x7f54041f02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:30.783 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.773+0000 7f5403fff700 1 -- 172.21.15.23:0/3737390731 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f4009550 con 0x7f540406b440 2023-03-27T10:02:30.783 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.773+0000 7f54037fe700 1 --2- 172.21.15.23:0/3737390731 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54040c6990 0x7f5404157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-27T10:02:30.783 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.773+0000 7f5403fff700 1 --2- 172.21.15.23:0/3737390731 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f540406b440 0x7f54041f2420 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f53f4000c00 tx=0x7f53f4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:30.784 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.774+0000 7f5400ff9700 1 -- 172.21.15.23:0/3737390731 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f4004480 con 0x7f540406b440 2023-03-27T10:02:30.784 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.774+0000 7f5400ff9700 1 -- 172.21.15.23:0/3737390731 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53f401f070 con 0x7f540406b440 2023-03-27T10:02:30.784 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.774+0000 7f540a9ff700 1 -- 172.21.15.23:0/3737390731 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5404158ce0 con 0x7f540406b440 2023-03-27T10:02:30.784 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.774+0000 7f5400ff9700 1 -- 172.21.15.23:0/3737390731 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f40038e0 con 0x7f540406b440 2023-03-27T10:02:30.784 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.774+0000 7f540a9ff700 1 -- 172.21.15.23:0/3737390731 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5404158a00 con 0x7f540406b440 2023-03-27T10:02:30.784 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.775+0000 7f540a9ff700 1 -- 172.21.15.23:0/3737390731 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5404158fc0 con 0x7f540406b440 2023-03-27T10:02:30.784 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.776+0000 7f5400ff9700 1 -- 172.21.15.23:0/3737390731 <== mon.0 v2:172.21.15.23:3300/0 4 ==== mgrmap(e 6) v1 ==== 85113+0+0 (secure 0 0 0) 0x7f53f402a020 con 0x7f540406b440 2023-03-27T10:02:30.784 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.777+0000 7f5400ff9700 1 --2- 172.21.15.23:0/3737390731 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f53f0066e60 0x7f53f0069310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:30.785 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.777+0000 7f5400ff9700 1 -- 172.21.15.23:0/3737390731 <== mon.0 v2:172.21.15.23:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f53f4085b00 con 0x7f540406b440 2023-03-27T10:02:30.785 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.777+0000 7f54037fe700 1 --2- 172.21.15.23:0/3737390731 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f53f0066e60 0x7f53f0069310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:30.785 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.778+0000 7f54037fe700 1 --2- 172.21.15.23:0/3737390731 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f53f0066e60 0x7f53f0069310 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f54040cda30 tx=0x7f53f8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:30.785 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.781+0000 7f5400ff9700 1 -- 172.21.15.23:0/3737390731 <== mon.0 v2:172.21.15.23:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173280 (secure 0 0 0) 0x7f53f4087e60 con 0x7f540406b440 2023-03-27T10:02:30.983 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.983+0000 7f540a9ff700 1 -- 172.21.15.23:0/3737390731 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f540414fd50 con 0x7f540406b440 2023-03-27T10:02:30.983 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.983+0000 7f5400ff9700 1 -- 172.21.15.23:0/3737390731 <== mon.0 v2:172.21.15.23:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f53f40150a0 con 0x7f540406b440 2023-03-27T10:02:30.984 INFO:teuthology.orchestra.run.smithi023.stdout:pg_num: 64 2023-03-27T10:02:30.986 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.985+0000 7f540a9ff700 1 -- 172.21.15.23:0/3737390731 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f53f0066e60 msgr2=0x7f53f0069310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:30.986 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.985+0000 7f540a9ff700 1 --2- 172.21.15.23:0/3737390731 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f53f0066e60 0x7f53f0069310 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f54040cda30 tx=0x7f53f8009040 comp rx=0 tx=0).stop 2023-03-27T10:02:30.986 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.985+0000 7f540a9ff700 1 -- 172.21.15.23:0/3737390731 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f540406b440 msgr2=0x7f54041f2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:30.987 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.985+0000 7f540a9ff700 1 --2- 172.21.15.23:0/3737390731 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f540406b440 0x7f54041f2420 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f53f4000c00 tx=0x7f53f4004750 comp rx=0 tx=0).stop 2023-03-27T10:02:30.987 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.986+0000 7f540a9ff700 1 -- 172.21.15.23:0/3737390731 shutdown_connections 2023-03-27T10:02:30.987 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.986+0000 7f540a9ff700 1 --2- 172.21.15.23:0/3737390731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54040c9520 0x7f54041f02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:30.987 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.986+0000 7f540a9ff700 1 --2- 172.21.15.23:0/3737390731 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f540406b440 0x7f54041f2420 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:30.988 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.986+0000 7f540a9ff700 1 --2- 172.21.15.23:0/3737390731 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f53f0066e60 0x7f53f0069310 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:30.988 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.986+0000 7f540a9ff700 1 --2- 172.21.15.23:0/3737390731 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54040c6990 0x7f5404157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:30.988 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.986+0000 7f540a9ff700 1 -- 172.21.15.23:0/3737390731 >> 172.21.15.23:0/3737390731 conn(0x7f5404071920 msgr2=0x7f54040cb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:30.989 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.986+0000 7f540a9ff700 1 -- 172.21.15.23:0/3737390731 shutdown_connections 2023-03-27T10:02:30.989 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:30.986+0000 7f540a9ff700 1 -- 172.21.15.23:0/3737390731 wait complete. 2023-03-27T10:02:31.002 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-03-27T10:02:31.197 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.198+0000 7fd7b69ff700 1 Processor -- start 2023-03-27T10:02:31.198 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.198+0000 7fd7b69ff700 1 -- start start 2023-03-27T10:02:31.198 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.199+0000 7fd7b69ff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd7b00c6990 0x7fd7b00c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:31.199 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.199+0000 7fd7b69ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd7b00c9520 0x7fd7b00c8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:31.199 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.199+0000 7fd7b69ff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7b00ca730 con 0x7fd7b006b440 2023-03-27T10:02:31.199 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.199+0000 7fd7b69ff700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7b00c72a0 con 0x7fd7b00c6990 2023-03-27T10:02:31.199 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.199+0000 7fd7b69ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b00cf9f0 con 0x7fd7b00c9520 2023-03-27T10:02:31.199 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.199+0000 7fd7a7fff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd7b00c9520 0x7fd7b00c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:31.199 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.199+0000 7fd7affff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd7b00c6990 0x7fd7b00c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:31.200 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.199+0000 7fd7b4f9c700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fd7b006b440 0x7fd7b006b880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.23:34482/0 (socket says 172.21.15.23:34482) 2023-03-27T10:02:31.200 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.199+0000 7fd7b4f9c700 1 -- 172.21.15.23:0/2722085875 learned_addr learned my addr 172.21.15.23:0/2722085875 (peer_addr_for_me v1:172.21.15.23:0/0) 2023-03-27T10:02:31.201 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.200+0000 7fd7af7fe700 1 -- 172.21.15.23:0/2722085875 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3966210204 0 0) 0x7fd7b00ca730 con 0x7fd7b006b440 2023-03-27T10:02:31.201 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.200+0000 7fd7a7fff700 1 -- 172.21.15.23:0/2722085875 >> v2:172.21.15.112:3300/0 conn(0x7fd7b00c6990 msgr2=0x7fd7b00c6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:02:31.201 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.200+0000 7fd7a7fff700 1 --2- 172.21.15.23:0/2722085875 >> v2:172.21.15.112:3300/0 conn(0x7fd7b00c6990 0x7fd7b00c6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:31.201 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.200+0000 7fd7a7fff700 1 -- 172.21.15.23:0/2722085875 >> v1:172.21.15.112:6789/0 conn(0x7fd7b006b440 legacy=0x7fd7b006b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:31.201 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.200+0000 7fd7a7fff700 1 -- 172.21.15.23:0/2722085875 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b0158a00 con 0x7fd7b00c9520 2023-03-27T10:02:31.202 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.200+0000 7fd7affff700 1 --2- 172.21.15.23:0/2722085875 >> v2:172.21.15.112:3300/0 conn(0x7fd7b00c6990 0x7fd7b00c6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-27T10:02:31.202 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.200+0000 7fd7a7fff700 1 --2- 172.21.15.23:0/2722085875 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd7b00c9520 0x7fd7b00c8820 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd798009840 tx=0x7fd798009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64896b4ee1a3df8f server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:31.202 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.200+0000 7fd7af7fe700 1 -- 172.21.15.23:0/2722085875 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd798011830 con 0x7fd7b00c9520 2023-03-27T10:02:31.202 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.200+0000 7fd7af7fe700 1 -- 172.21.15.23:0/2722085875 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd798011990 con 0x7fd7b00c9520 2023-03-27T10:02:31.202 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.200+0000 7fd7af7fe700 1 -- 172.21.15.23:0/2722085875 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd798011d20 con 0x7fd7b00c9520 2023-03-27T10:02:31.202 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.201+0000 7fd7b69ff700 1 -- 172.21.15.23:0/2722085875 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd7b00c9520 msgr2=0x7fd7b00c8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:31.202 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.201+0000 7fd7b69ff700 1 --2- 172.21.15.23:0/2722085875 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd7b00c9520 0x7fd7b00c8820 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd798009840 tx=0x7fd798009b50 comp rx=0 tx=0).stop 2023-03-27T10:02:31.203 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.201+0000 7fd7b69ff700 1 -- 172.21.15.23:0/2722085875 shutdown_connections 2023-03-27T10:02:31.203 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.201+0000 7fd7b69ff700 1 --2- 172.21.15.23:0/2722085875 >> v2:172.21.15.112:3300/0 conn(0x7fd7b00c6990 0x7fd7b00c6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:31.203 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.201+0000 7fd7b69ff700 1 --2- 172.21.15.23:0/2722085875 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd7b00c9520 0x7fd7b00c8820 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:31.203 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.201+0000 7fd7b69ff700 1 -- 172.21.15.23:0/2722085875 >> 172.21.15.23:0/2722085875 conn(0x7fd7b0071920 msgr2=0x7fd7b0071cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:31.203 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.201+0000 7fd7b69ff700 1 -- 172.21.15.23:0/2722085875 shutdown_connections 2023-03-27T10:02:31.203 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.201+0000 7fd7b69ff700 1 -- 172.21.15.23:0/2722085875 wait complete. 2023-03-27T10:02:31.205 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.203+0000 7fd7b69ff700 1 Processor -- start 2023-03-27T10:02:31.205 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.203+0000 7fd7b69ff700 1 -- start start 2023-03-27T10:02:31.205 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.204+0000 7fd7b69ff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fd7b006b440 0x7fd7b014db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:31.205 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.204+0000 7fd7b69ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd7b00c6990 0x7fd7b01f22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:31.206 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.204+0000 7fd7b69ff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7b00c9520 0x7fd7b01556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:31.209 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.204+0000 7fd7b69ff700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b01504c0 con 0x7fd7b006b440 2023-03-27T10:02:31.210 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.204+0000 7fd7b69ff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b0150760 con 0x7fd7b00c9520 2023-03-27T10:02:31.210 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.204+0000 7fd7b69ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b014e0c0 con 0x7fd7b00c6990 2023-03-27T10:02:31.210 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.204+0000 7fd7affff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fd7b006b440 0x7fd7b014db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:31.210 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.204+0000 7fd7affff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fd7b006b440 0x7fd7b014db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:42466/0 (socket says 172.21.15.23:42466) 2023-03-27T10:02:31.210 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.204+0000 7fd7affff700 1 -- 172.21.15.23:0/2861922870 learned_addr learned my addr 172.21.15.23:0/2861922870 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:31.211 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.204+0000 7fd7a7fff700 1 --2- 172.21.15.23:0/2861922870 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd7b00c6990 0x7fd7b01f22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:31.211 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.204+0000 7fd7b4f9c700 1 --2- 172.21.15.23:0/2861922870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7b00c9520 0x7fd7b01556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:31.211 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.204+0000 7fd7affff700 1 -- 172.21.15.23:0/2861922870 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd7b00c6990 msgr2=0x7fd7b01f22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:31.211 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.204+0000 7fd7affff700 1 --2- 172.21.15.23:0/2861922870 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd7b00c6990 0x7fd7b01f22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:31.211 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.204+0000 7fd7affff700 1 -- 172.21.15.23:0/2861922870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7b00c9520 msgr2=0x7fd7b01556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:31.211 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.204+0000 7fd7affff700 1 --2- 172.21.15.23:0/2861922870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7b00c9520 0x7fd7b01556e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:31.211 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.204+0000 7fd7affff700 1 -- 172.21.15.23:0/2861922870 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a00094a0 con 0x7fd7b006b440 2023-03-27T10:02:31.211 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.205+0000 7fd7affff700 1 --2- 172.21.15.23:0/2861922870 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fd7b006b440 0x7fd7b014db80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd7a000ebe0 tx=0x7fd7a000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:31.212 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.205+0000 7fd7ad7fa700 1 -- 172.21.15.23:0/2861922870 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a000c850 con 0x7fd7b006b440 2023-03-27T10:02:31.212 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.205+0000 7fd7ad7fa700 1 -- 172.21.15.23:0/2861922870 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a000c9b0 con 0x7fd7b006b440 2023-03-27T10:02:31.212 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.205+0000 7fd7b69ff700 1 -- 172.21.15.23:0/2861922870 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7980094a0 con 0x7fd7b006b440 2023-03-27T10:02:31.212 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.205+0000 7fd7ad7fa700 1 -- 172.21.15.23:0/2861922870 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a000cd40 con 0x7fd7b006b440 2023-03-27T10:02:31.212 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.205+0000 7fd7b69ff700 1 -- 172.21.15.23:0/2861922870 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b0158fc0 con 0x7fd7b006b440 2023-03-27T10:02:31.213 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.206+0000 7fd7b69ff700 1 -- 172.21.15.23:0/2861922870 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7b0158ce0 con 0x7fd7b006b440 2023-03-27T10:02:31.213 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.207+0000 7fd7ad7fa700 1 -- 172.21.15.23:0/2861922870 <== mon.0 v2:172.21.15.23:3300/0 4 ==== mgrmap(e 6) v1 ==== 85113+0+0 (secure 0 0 0) 0x7fd7a0019030 con 0x7fd7b006b440 2023-03-27T10:02:31.214 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.208+0000 7fd7ad7fa700 1 --2- 172.21.15.23:0/2861922870 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fd790066e60 0x7fd790069310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:31.214 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.208+0000 7fd7ad7fa700 1 -- 172.21.15.23:0/2861922870 <== mon.0 v2:172.21.15.23:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd7a00844c0 con 0x7fd7b006b440 2023-03-27T10:02:31.214 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.208+0000 7fd7a7fff700 1 --2- 172.21.15.23:0/2861922870 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fd790066e60 0x7fd790069310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:31.214 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.209+0000 7fd7a7fff700 1 --2- 172.21.15.23:0/2861922870 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fd790066e60 0x7fd790069310 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd7980090b0 tx=0x7fd798008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:31.214 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.212+0000 7fd7ad7fa700 1 -- 172.21.15.23:0/2861922870 <== mon.0 v2:172.21.15.23:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173280 (secure 0 0 0) 0x7fd7a0051410 con 0x7fd7b006b440 2023-03-27T10:02:31.416 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.417+0000 7fd7b69ff700 1 -- 172.21.15.23:0/2861922870 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd7b01592a0 con 0x7fd7b006b440 2023-03-27T10:02:31.417 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.417+0000 7fd7ad7fa700 1 -- 172.21.15.23:0/2861922870 <== mon.0 v2:172.21.15.23:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd7a000cea0 con 0x7fd7b006b440 2023-03-27T10:02:31.417 INFO:teuthology.orchestra.run.smithi023.stdout:pg_num: 32 2023-03-27T10:02:31.419 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.420+0000 7fd7b69ff700 1 -- 172.21.15.23:0/2861922870 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fd790066e60 msgr2=0x7fd790069310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:31.420 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.420+0000 7fd7b69ff700 1 --2- 172.21.15.23:0/2861922870 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fd790066e60 0x7fd790069310 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd7980090b0 tx=0x7fd798008030 comp rx=0 tx=0).stop 2023-03-27T10:02:31.420 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.420+0000 7fd7b69ff700 1 -- 172.21.15.23:0/2861922870 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fd7b006b440 msgr2=0x7fd7b014db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:31.420 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.420+0000 7fd7b69ff700 1 --2- 172.21.15.23:0/2861922870 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fd7b006b440 0x7fd7b014db80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd7a000ebe0 tx=0x7fd7a000eef0 comp rx=0 tx=0).stop 2023-03-27T10:02:31.420 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.420+0000 7fd7b69ff700 1 -- 172.21.15.23:0/2861922870 shutdown_connections 2023-03-27T10:02:31.420 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.420+0000 7fd7b69ff700 1 --2- 172.21.15.23:0/2861922870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7b00c9520 0x7fd7b01556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:31.421 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.420+0000 7fd7b69ff700 1 --2- 172.21.15.23:0/2861922870 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fd7b006b440 0x7fd7b014db80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:31.421 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.420+0000 7fd7b69ff700 1 --2- 172.21.15.23:0/2861922870 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7fd790066e60 0x7fd790069310 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:31.421 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.420+0000 7fd7b69ff700 1 --2- 172.21.15.23:0/2861922870 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd7b00c6990 0x7fd7b01f22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:31.421 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.420+0000 7fd7b69ff700 1 -- 172.21.15.23:0/2861922870 >> 172.21.15.23:0/2861922870 conn(0x7fd7b0071920 msgr2=0x7fd7b0156080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:31.421 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.420+0000 7fd7b69ff700 1 -- 172.21.15.23:0/2861922870 shutdown_connections 2023-03-27T10:02:31.421 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.420+0000 7fd7b69ff700 1 -- 172.21.15.23:0/2861922870 wait complete. 2023-03-27T10:02:31.438 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph 'osd blocklist ls' 2023-03-27T10:02:31.627 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.627+0000 7f44abbff700 1 Processor -- start 2023-03-27T10:02:31.627 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.627+0000 7f44abbff700 1 -- start start 2023-03-27T10:02:31.630 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.628+0000 7f44abbff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f44a40c6990 0x7f44a40c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:31.631 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.628+0000 7f44abbff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f44a406b440 0x7f44a406b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:31.631 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.628+0000 7f44abbff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44a40ca730 con 0x7f44a40c9520 2023-03-27T10:02:31.631 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.628+0000 7f44abbff700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44a40c72a0 con 0x7f44a406b440 2023-03-27T10:02:31.631 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.628+0000 7f44abbff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a40cf9f0 con 0x7f44a40c6990 2023-03-27T10:02:31.632 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.629+0000 7f44a999b700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f44a40c6990 0x7f44a40c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:31.632 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.629+0000 7f44aa19c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f44a406b440 0x7f44a406b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:31.632 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.629+0000 7f44a919a700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f44a40c9520 0x7f44a40c8820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.23:34490/0 (socket says 172.21.15.23:34490) 2023-03-27T10:02:31.633 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.629+0000 7f44a919a700 1 -- 172.21.15.23:0/4158008165 learned_addr learned my addr 172.21.15.23:0/4158008165 (peer_addr_for_me v1:172.21.15.23:0/0) 2023-03-27T10:02:31.634 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.629+0000 7f44aa19c700 1 -- 172.21.15.23:0/4158008165 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f44a40c6990 msgr2=0x7f44a40c6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:02:31.634 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.629+0000 7f44aa19c700 1 --2- 172.21.15.23:0/4158008165 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f44a40c6990 0x7f44a40c6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:31.634 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.629+0000 7f44a8999700 1 -- 172.21.15.23:0/4158008165 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3988057801 0 0) 0x7f44a40ca730 con 0x7f44a40c9520 2023-03-27T10:02:31.634 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.629+0000 7f44aa19c700 1 -- 172.21.15.23:0/4158008165 >> v1:172.21.15.112:6789/0 conn(0x7f44a40c9520 legacy=0x7f44a40c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:31.635 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.629+0000 7f44aa19c700 1 -- 172.21.15.23:0/4158008165 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a40c8f30 con 0x7f44a406b440 2023-03-27T10:02:31.635 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.629+0000 7f44a999b700 1 --2- 172.21.15.23:0/4158008165 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f44a40c6990 0x7f44a40c6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-27T10:02:31.635 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.629+0000 7f44aa19c700 1 --2- 172.21.15.23:0/4158008165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44a406b440 0x7f44a406b860 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f449c009d90 tx=0x7f449c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34a0a00ae8549d76 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:31.635 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.630+0000 7f44a8999700 1 -- 172.21.15.23:0/4158008165 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449c00e070 con 0x7f44a406b440 2023-03-27T10:02:31.635 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.630+0000 7f44a8999700 1 -- 172.21.15.23:0/4158008165 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f449c00bb20 con 0x7f44a406b440 2023-03-27T10:02:31.635 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.630+0000 7f44a8999700 1 -- 172.21.15.23:0/4158008165 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449c012430 con 0x7f44a406b440 2023-03-27T10:02:31.635 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.630+0000 7f44abbff700 1 -- 172.21.15.23:0/4158008165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44a406b440 msgr2=0x7f44a406b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:31.636 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.630+0000 7f44abbff700 1 --2- 172.21.15.23:0/4158008165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44a406b440 0x7f44a406b860 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f449c009d90 tx=0x7f449c00b330 comp rx=0 tx=0).stop 2023-03-27T10:02:31.636 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.630+0000 7f44abbff700 1 -- 172.21.15.23:0/4158008165 shutdown_connections 2023-03-27T10:02:31.636 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.630+0000 7f44abbff700 1 --2- 172.21.15.23:0/4158008165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44a406b440 0x7f44a406b860 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:31.636 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.630+0000 7f44abbff700 1 --2- 172.21.15.23:0/4158008165 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f44a40c6990 0x7f44a40c6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:31.636 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.630+0000 7f44abbff700 1 -- 172.21.15.23:0/4158008165 >> 172.21.15.23:0/4158008165 conn(0x7f44a4071920 msgr2=0x7f44a4071cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:31.636 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.630+0000 7f44abbff700 1 -- 172.21.15.23:0/4158008165 shutdown_connections 2023-03-27T10:02:31.636 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.630+0000 7f44abbff700 1 -- 172.21.15.23:0/4158008165 wait complete. 2023-03-27T10:02:31.639 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.632+0000 7f44abbff700 1 Processor -- start 2023-03-27T10:02:31.639 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.632+0000 7f44abbff700 1 -- start start 2023-03-27T10:02:31.640 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.633+0000 7f44abbff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44a406b440 0x7f44a41f0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:31.640 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.633+0000 7f44abbff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f44a40c6990 0x7f44a41540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:31.640 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.633+0000 7f44abbff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f44a40c9520 0x7f44a41e22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:31.641 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.633+0000 7f44abbff700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a41f2160 con 0x7f44a40c9520 2023-03-27T10:02:31.641 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.633+0000 7f44abbff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a41e27f0 con 0x7f44a406b440 2023-03-27T10:02:31.641 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.633+0000 7f44abbff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a41efdc0 con 0x7f44a40c6990 2023-03-27T10:02:31.641 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.633+0000 7f44aa19c700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f44a40c9520 0x7f44a41e22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:31.641 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.633+0000 7f44aa19c700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f44a40c9520 0x7f44a41e22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:42474/0 (socket says 172.21.15.23:42474) 2023-03-27T10:02:31.641 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.633+0000 7f44aa19c700 1 -- 172.21.15.23:0/4064639671 learned_addr learned my addr 172.21.15.23:0/4064639671 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:31.642 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.633+0000 7f44a919a700 1 --2- 172.21.15.23:0/4064639671 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f44a40c6990 0x7f44a41540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:31.642 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.633+0000 7f44a999b700 1 --2- 172.21.15.23:0/4064639671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44a406b440 0x7f44a41f0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:31.642 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.634+0000 7f44aa19c700 1 -- 172.21.15.23:0/4064639671 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f44a40c6990 msgr2=0x7f44a41540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:31.642 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.634+0000 7f44aa19c700 1 --2- 172.21.15.23:0/4064639671 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f44a40c6990 0x7f44a41540a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:31.642 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.634+0000 7f44aa19c700 1 -- 172.21.15.23:0/4064639671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44a406b440 msgr2=0x7f44a41f0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:31.642 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.634+0000 7f44aa19c700 1 --2- 172.21.15.23:0/4064639671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44a406b440 0x7f44a41f0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:31.642 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.634+0000 7f44aa19c700 1 -- 172.21.15.23:0/4064639671 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a00094a0 con 0x7f44a40c9520 2023-03-27T10:02:31.642 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.634+0000 7f44aa19c700 1 --2- 172.21.15.23:0/4064639671 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f44a40c9520 0x7f44a41e22b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f44a4072540 tx=0x7f449c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:31.643 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.634+0000 7f449a7fc700 1 -- 172.21.15.23:0/4064639671 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449c012430 con 0x7f44a40c9520 2023-03-27T10:02:31.643 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.634+0000 7f449a7fc700 1 -- 172.21.15.23:0/4064639671 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f449c020070 con 0x7f44a40c9520 2023-03-27T10:02:31.643 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.634+0000 7f449a7fc700 1 -- 172.21.15.23:0/4064639671 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449c004280 con 0x7f44a40c9520 2023-03-27T10:02:31.643 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.634+0000 7f44abbff700 1 -- 172.21.15.23:0/4064639671 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f449c009a70 con 0x7f44a40c9520 2023-03-27T10:02:31.643 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.634+0000 7f44abbff700 1 -- 172.21.15.23:0/4064639671 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44a4158ce0 con 0x7f44a40c9520 2023-03-27T10:02:31.643 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.635+0000 7f44abbff700 1 -- 172.21.15.23:0/4064639671 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44a4158a00 con 0x7f44a40c9520 2023-03-27T10:02:31.643 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.636+0000 7f449a7fc700 1 -- 172.21.15.23:0/4064639671 <== mon.0 v2:172.21.15.23:3300/0 4 ==== mgrmap(e 6) v1 ==== 85113+0+0 (secure 0 0 0) 0x7f449c02e070 con 0x7f44a40c9520 2023-03-27T10:02:31.644 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.637+0000 7f449a7fc700 1 --2- 172.21.15.23:0/4064639671 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f4488066e30 0x7f44880692e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:31.644 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.637+0000 7f449a7fc700 1 -- 172.21.15.23:0/4064639671 <== mon.0 v2:172.21.15.23:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f449c022070 con 0x7f44a40c9520 2023-03-27T10:02:31.644 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.637+0000 7f44a999b700 1 --2- 172.21.15.23:0/4064639671 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f4488066e30 0x7f44880692e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:31.645 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.638+0000 7f44a999b700 1 --2- 172.21.15.23:0/4064639671 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7f4488066e30 0x7f44880692e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f44a0009d70 tx=0x7f44a0009210 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:31.645 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:31.641+0000 7f449a7fc700 1 -- 172.21.15.23:0/4064639671 <== mon.0 v2:172.21.15.23:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173280 (secure 0 0 0) 0x7f449c053120 con 0x7f44a40c9520 2023-03-27T10:02:32.090 INFO:teuthology.orchestra.run.smithi023.stderr:no valid command found; 10 closest matches: 2023-03-27T10:02:32.091 INFO:teuthology.orchestra.run.smithi023.stderr:pg stat 2023-03-27T10:02:32.091 INFO:teuthology.orchestra.run.smithi023.stderr:pg getmap 2023-03-27T10:02:32.091 INFO:teuthology.orchestra.run.smithi023.stderr:pg dump [...] 2023-03-27T10:02:32.091 INFO:teuthology.orchestra.run.smithi023.stderr:pg dump_json [...] 2023-03-27T10:02:32.092 INFO:teuthology.orchestra.run.smithi023.stderr:pg dump_pools_json 2023-03-27T10:02:32.092 INFO:teuthology.orchestra.run.smithi023.stderr:pg ls-by-pool [...] 2023-03-27T10:02:32.092 INFO:teuthology.orchestra.run.smithi023.stderr:pg ls-by-primary [] [...] 2023-03-27T10:02:32.093 INFO:teuthology.orchestra.run.smithi023.stderr:pg ls-by-osd [] [...] 2023-03-27T10:02:32.093 INFO:teuthology.orchestra.run.smithi023.stderr:pg ls [] [...] 2023-03-27T10:02:32.094 INFO:teuthology.orchestra.run.smithi023.stderr:pg dump_stuck [...] [] 2023-03-27T10:02:32.094 INFO:teuthology.orchestra.run.smithi023.stderr:Error EINVAL: invalid command 2023-03-27T10:02:32.097 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-03-27T10:02:32.098 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph 'osd blacklist ls' 2023-03-27T10:02:32.285 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.285+0000 7ff0d89ff700 1 Processor -- start 2023-03-27T10:02:32.285 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.285+0000 7ff0d89ff700 1 -- start start 2023-03-27T10:02:32.286 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.287+0000 7ff0d89ff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff0d4152de0 0x7ff0d406bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:32.286 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.287+0000 7ff0d89ff700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7ff0d40c9510 0x7ff0d40c4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:32.286 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.287+0000 7ff0d89ff700 1 -- --> v1:172.21.15.23:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0d406c210 con 0x7ff0d40b4410 2023-03-27T10:02:32.287 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.287+0000 7ff0d89ff700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0d41531b0 con 0x7ff0d40c9510 2023-03-27T10:02:32.287 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.287+0000 7ff0d89ff700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0d415cb20 con 0x7ff0d4152de0 2023-03-27T10:02:32.288 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.287+0000 7ff0cbfff700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7ff0d40c9510 0x7ff0d40c4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:32.289 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.287+0000 7ff0d2d9d700 1 --1- >> v1:172.21.15.23:6789/0 conn(0x7ff0d40b4410 0x7ff0d40b2360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.23:6789/0 says I am v1:172.21.15.23:59284/0 (socket says 172.21.15.23:59284) 2023-03-27T10:02:32.289 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.287+0000 7ff0d2d9d700 1 -- 172.21.15.23:0/3534424573 learned_addr learned my addr 172.21.15.23:0/3534424573 (peer_addr_for_me v1:172.21.15.23:0/0) 2023-03-27T10:02:32.289 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.287+0000 7ff0d259c700 1 --2- 172.21.15.23:0/3534424573 >> v2:172.21.15.112:3300/0 conn(0x7ff0d4152de0 0x7ff0d406bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:32.289 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.288+0000 7ff0cbfff700 1 -- 172.21.15.23:0/3534424573 >> v2:172.21.15.112:3300/0 conn(0x7ff0d4152de0 msgr2=0x7ff0d406bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-27T10:02:32.290 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.288+0000 7ff0cbfff700 1 --2- 172.21.15.23:0/3534424573 >> v2:172.21.15.112:3300/0 conn(0x7ff0d4152de0 0x7ff0d406bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:32.290 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.288+0000 7ff0cbfff700 1 -- 172.21.15.23:0/3534424573 >> v1:172.21.15.23:6789/0 conn(0x7ff0d40b4410 legacy=0x7ff0d40b2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:32.290 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.288+0000 7ff0cbfff700 1 -- 172.21.15.23:0/3534424573 --> v2:172.21.15.23:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d40c99e0 con 0x7ff0d40c9510 2023-03-27T10:02:32.291 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.288+0000 7ff0d1d9b700 1 -- 172.21.15.23:0/3534424573 <== mon.0 v1:172.21.15.23:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3124228332 0 0) 0x7ff0d406c210 con 0x7ff0d40b4410 2023-03-27T10:02:32.291 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.288+0000 7ff0cbfff700 1 --2- 172.21.15.23:0/3534424573 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7ff0d40c9510 0x7ff0d40c4270 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff0d408c050 tx=0x7ff0bc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d936924636a3cb2d server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:32.291 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.288+0000 7ff0d1d9b700 1 -- 172.21.15.23:0/3534424573 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc011830 con 0x7ff0d40c9510 2023-03-27T10:02:32.291 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.288+0000 7ff0d1d9b700 1 -- 172.21.15.23:0/3534424573 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0bc011990 con 0x7ff0d40c9510 2023-03-27T10:02:32.291 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.288+0000 7ff0d1d9b700 1 -- 172.21.15.23:0/3534424573 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc011d20 con 0x7ff0d40c9510 2023-03-27T10:02:32.291 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.289+0000 7ff0d89ff700 1 -- 172.21.15.23:0/3534424573 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7ff0d40c9510 msgr2=0x7ff0d40c4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:32.292 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.289+0000 7ff0d89ff700 1 --2- 172.21.15.23:0/3534424573 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7ff0d40c9510 0x7ff0d40c4270 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff0d408c050 tx=0x7ff0bc009b20 comp rx=0 tx=0).stop 2023-03-27T10:02:32.292 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.289+0000 7ff0d89ff700 1 -- 172.21.15.23:0/3534424573 shutdown_connections 2023-03-27T10:02:32.292 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.289+0000 7ff0d89ff700 1 --2- 172.21.15.23:0/3534424573 >> v2:172.21.15.112:3300/0 conn(0x7ff0d4152de0 0x7ff0d406bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:32.292 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.289+0000 7ff0d89ff700 1 --2- 172.21.15.23:0/3534424573 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7ff0d40c9510 0x7ff0d40c4270 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:32.293 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.289+0000 7ff0d89ff700 1 -- 172.21.15.23:0/3534424573 >> 172.21.15.23:0/3534424573 conn(0x7ff0d4065ca0 msgr2=0x7ff0d40660a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-27T10:02:32.293 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.289+0000 7ff0d89ff700 1 -- 172.21.15.23:0/3534424573 shutdown_connections 2023-03-27T10:02:32.293 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.289+0000 7ff0d89ff700 1 -- 172.21.15.23:0/3534424573 wait complete. 2023-03-27T10:02:32.295 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.291+0000 7ff0d89ff700 1 Processor -- start 2023-03-27T10:02:32.295 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.292+0000 7ff0d89ff700 1 -- start start 2023-03-27T10:02:32.296 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.293+0000 7ff0d89ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0d40b4410 0x7ff0d4148980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:32.296 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.293+0000 7ff0d89ff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7ff0d40c9510 0x7ff0d414b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:32.301 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.293+0000 7ff0d89ff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0d4152de0 0x7ff0d414a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:32.301 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.293+0000 7ff0d89ff700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d414af80 con 0x7ff0d40c9510 2023-03-27T10:02:32.301 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.293+0000 7ff0d89ff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d414a250 con 0x7ff0d4152de0 2023-03-27T10:02:32.301 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.293+0000 7ff0d89ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d4149c40 con 0x7ff0d40b4410 2023-03-27T10:02:32.301 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.293+0000 7ff0cbfff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7ff0d40c9510 0x7ff0d414b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:32.302 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.293+0000 7ff0cbfff700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7ff0d40c9510 0x7ff0d414b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:42492/0 (socket says 172.21.15.23:42492) 2023-03-27T10:02:32.302 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.293+0000 7ff0cbfff700 1 -- 172.21.15.23:0/4126664502 learned_addr learned my addr 172.21.15.23:0/4126664502 (peer_addr_for_me v2:172.21.15.23:0/0) 2023-03-27T10:02:32.302 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.293+0000 7ff0d259c700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0d40b4410 0x7ff0d4148980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:32.302 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.293+0000 7ff0d2d9d700 1 --2- 172.21.15.23:0/4126664502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0d4152de0 0x7ff0d414a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:32.302 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.293+0000 7ff0cbfff700 1 -- 172.21.15.23:0/4126664502 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0d40b4410 msgr2=0x7ff0d4148980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:32.302 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.293+0000 7ff0cbfff700 1 --2- 172.21.15.23:0/4126664502 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0d40b4410 0x7ff0d4148980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:32.302 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.293+0000 7ff0cbfff700 1 -- 172.21.15.23:0/4126664502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0d4152de0 msgr2=0x7ff0d414a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-27T10:02:32.303 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.293+0000 7ff0cbfff700 1 --2- 172.21.15.23:0/4126664502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0d4152de0 0x7ff0d414a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-27T10:02:32.303 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.293+0000 7ff0cbfff700 1 -- 172.21.15.23:0/4126664502 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0bc0094a0 con 0x7ff0d40c9510 2023-03-27T10:02:32.303 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.294+0000 7ff0cbfff700 1 --2- 172.21.15.23:0/4126664502 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7ff0d40c9510 0x7ff0d414b4b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc0090b0 tx=0x7ff0bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:32.303 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.294+0000 7ff0caffd700 1 -- 172.21.15.23:0/4126664502 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc0036a0 con 0x7ff0d40c9510 2023-03-27T10:02:32.303 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.294+0000 7ff0caffd700 1 -- 172.21.15.23:0/4126664502 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0bc01f070 con 0x7ff0d40c9510 2023-03-27T10:02:32.303 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.294+0000 7ff0d89ff700 1 -- 172.21.15.23:0/4126664502 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d415c1f0 con 0x7ff0d40c9510 2023-03-27T10:02:32.303 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.294+0000 7ff0caffd700 1 -- 172.21.15.23:0/4126664502 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc003a20 con 0x7ff0d40c9510 2023-03-27T10:02:32.303 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.294+0000 7ff0d89ff700 1 -- 172.21.15.23:0/4126664502 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d415bf10 con 0x7ff0d40c9510 2023-03-27T10:02:32.304 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.295+0000 7ff0d89ff700 1 -- 172.21.15.23:0/4126664502 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0d415c4d0 con 0x7ff0d40c9510 2023-03-27T10:02:32.304 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.296+0000 7ff0caffd700 1 -- 172.21.15.23:0/4126664502 <== mon.0 v2:172.21.15.23:3300/0 4 ==== mgrmap(e 6) v1 ==== 85113+0+0 (secure 0 0 0) 0x7ff0bc00f050 con 0x7ff0d40c9510 2023-03-27T10:02:32.305 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.297+0000 7ff0caffd700 1 --2- 172.21.15.23:0/4126664502 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7ff0b4066e60 0x7ff0b4069310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-27T10:02:32.305 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.297+0000 7ff0caffd700 1 -- 172.21.15.23:0/4126664502 <== mon.0 v2:172.21.15.23:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff0bc021070 con 0x7ff0d40c9510 2023-03-27T10:02:32.305 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.297+0000 7ff0d259c700 1 --2- 172.21.15.23:0/4126664502 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7ff0b4066e60 0x7ff0b4069310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-27T10:02:32.305 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.298+0000 7ff0d259c700 1 --2- 172.21.15.23:0/4126664502 >> [v2:172.21.15.112:6832/127332,v1:172.21.15.112:6833/127332] conn(0x7ff0b4066e60 0x7ff0b4069310 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4009b40 tx=0x7ff0c4009210 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-27T10:02:32.306 INFO:teuthology.orchestra.run.smithi023.stderr:2023-03-27T10:02:32.301+0000 7ff0caffd700 1 -- 172.21.15.23:0/4126664502 <== mon.0 v2:172.21.15.23:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173280 (secure 0 0 0) 0x7ff0bc052ce0 con 0x7ff0d40c9510 2023-03-27T10:02:32.744 INFO:teuthology.orchestra.run.smithi023.stderr:no valid command found; 10 closest matches: 2023-03-27T10:02:32.744 INFO:teuthology.orchestra.run.smithi023.stderr:pg stat 2023-03-27T10:02:32.744 INFO:teuthology.orchestra.run.smithi023.stderr:pg getmap 2023-03-27T10:02:32.744 INFO:teuthology.orchestra.run.smithi023.stderr:pg dump [...] 2023-03-27T10:02:32.744 INFO:teuthology.orchestra.run.smithi023.stderr:pg dump_json [...] 2023-03-27T10:02:32.745 INFO:teuthology.orchestra.run.smithi023.stderr:pg dump_pools_json 2023-03-27T10:02:32.745 INFO:teuthology.orchestra.run.smithi023.stderr:pg ls-by-pool [...] 2023-03-27T10:02:32.745 INFO:teuthology.orchestra.run.smithi023.stderr:pg ls-by-primary [] [...] 2023-03-27T10:02:32.745 INFO:teuthology.orchestra.run.smithi023.stderr:pg ls-by-osd [] [...] 2023-03-27T10:02:32.745 INFO:teuthology.orchestra.run.smithi023.stderr:pg ls [] [...] 2023-03-27T10:02:32.745 INFO:teuthology.orchestra.run.smithi023.stderr:pg dump_stuck [...] [] 2023-03-27T10:02:32.745 INFO:teuthology.orchestra.run.smithi023.stderr:Error EINVAL: invalid command 2023-03-27T10:02:32.752 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-03-27T10:02:32.753 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa/tasks/cephfs/mount.py", line 192, in is_blocked output = self.fs.get_ceph_cmd_stdout('osd blocklist ls') File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa/tasks/cephfs/filesystem.py", line 248, in get_ceph_cmd_stdout return self.mon_manager.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa/tasks/ceph_manager.py", line 1579, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi023 with status 22: "sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph 'osd blocklist ls'" During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/contextutil.py", line 31, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa/tasks/ceph.py", line 468, in cephfs_setup fs = Filesystem(ctx, fs_config=temp, name=name, create=True) File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa/tasks/cephfs/filesystem.py", line 538, in __init__ self.create() File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa/tasks/cephfs/filesystem.py", line 723, in create self.run_client_payload(f"setfattr -n ceph.dir.layout.pool -v {ec_data_pool_name} . && getfattr -n ceph.dir.layout .") File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa/tasks/cephfs/filesystem.py", line 798, in run_client_payload m.umount_wait(require_clean=True) File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa/tasks/cephfs/fuse_mount.py", line 389, in umount_wait if self.is_blocked(): File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa/tasks/cephfs/mount.py", line 195, in is_blocked output = self.fs.get_ceph_cmd_stdout('osd blacklist ls') File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa/tasks/cephfs/filesystem.py", line 248, in get_ceph_cmd_stdout return self.mon_manager.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa/tasks/ceph_manager.py", line 1579, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi023 with status 22: "sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph 'osd blacklist ls'" 2023-03-27T10:02:32.755 INFO:teuthology.misc:Shutting down mds daemons... 2023-03-27T10:02:32.756 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-03-27T10:02:32.757 INFO:teuthology.orchestra.run:waiting for 300 2023-03-27T10:02:32.771 INFO:tasks.ceph.mds.a:Stopped 2023-03-27T10:02:32.773 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-03-27T10:02:32.774 INFO:teuthology.orchestra.run:waiting for 300 2023-03-27T10:02:32.825 INFO:tasks.ceph.mds.c:Stopped 2023-03-27T10:02:32.826 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2023-03-27T10:02:32.826 INFO:teuthology.orchestra.run:waiting for 300 2023-03-27T10:02:32.878 INFO:tasks.ceph.mds.e:Stopped 2023-03-27T10:02:32.879 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-03-27T10:02:32.880 INFO:teuthology.orchestra.run:waiting for 300 2023-03-27T10:02:32.889 INFO:tasks.ceph.mds.b:Stopped 2023-03-27T10:02:32.890 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-03-27T10:02:32.890 INFO:teuthology.orchestra.run:waiting for 300 2023-03-27T10:02:32.940 INFO:tasks.ceph.mds.d:Stopped 2023-03-27T10:02:32.942 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2023-03-27T10:02:32.943 INFO:teuthology.orchestra.run:waiting for 300 2023-03-27T10:02:32.991 INFO:tasks.ceph.mds.f:Stopped 2023-03-27T10:02:32.992 INFO:teuthology.misc:Shutting down osd daemons... 2023-03-27T10:02:32.992 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-03-27T10:02:32.993 INFO:teuthology.orchestra.run:waiting for 300 2023-03-27T10:02:33.032 INFO:tasks.ceph.osd.0:Stopped 2023-03-27T10:02:33.033 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-03-27T10:02:33.034 INFO:teuthology.orchestra.run:waiting for 300 2023-03-27T10:02:33.119 INFO:tasks.ceph.osd.1:Stopped 2023-03-27T10:02:33.120 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-03-27T10:02:33.120 INFO:teuthology.orchestra.run:waiting for 300 2023-03-27T10:02:33.167 INFO:tasks.ceph.osd.2:Stopped 2023-03-27T10:02:33.168 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-03-27T10:02:33.169 INFO:teuthology.orchestra.run:waiting for 300 2023-03-27T10:02:33.211 INFO:tasks.ceph.osd.3:Stopped 2023-03-27T10:02:33.212 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-03-27T10:02:33.212 INFO:teuthology.orchestra.run:waiting for 300 2023-03-27T10:02:33.247 INFO:tasks.ceph.osd.4:Stopped 2023-03-27T10:02:33.248 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-03-27T10:02:33.249 INFO:teuthology.orchestra.run:waiting for 300 2023-03-27T10:02:33.324 INFO:tasks.ceph.osd.5:Stopped 2023-03-27T10:02:33.325 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-03-27T10:02:33.325 INFO:teuthology.orchestra.run:waiting for 300 2023-03-27T10:02:33.394 INFO:tasks.ceph.osd.6:Stopped 2023-03-27T10:02:33.395 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-03-27T10:02:33.396 INFO:teuthology.orchestra.run:waiting for 300 2023-03-27T10:02:33.476 INFO:tasks.ceph.osd.7:Stopped 2023-03-27T10:02:33.477 INFO:teuthology.misc:Shutting down mgr daemons... 2023-03-27T10:02:33.478 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-03-27T10:02:33.478 INFO:teuthology.orchestra.run:waiting for 300 2023-03-27T10:02:33.534 INFO:tasks.ceph.mgr.y:Stopped 2023-03-27T10:02:33.534 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-03-27T10:02:33.535 INFO:teuthology.orchestra.run:waiting for 300 2023-03-27T10:02:33.561 INFO:tasks.ceph.mgr.x:Stopped 2023-03-27T10:02:33.562 INFO:teuthology.misc:Shutting down mon daemons... 2023-03-27T10:02:33.562 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-03-27T10:02:33.563 INFO:teuthology.orchestra.run:waiting for 300 2023-03-27T10:02:33.580 INFO:tasks.ceph.mon.a:Stopped 2023-03-27T10:02:33.581 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-03-27T10:02:33.581 INFO:teuthology.orchestra.run:waiting for 300 2023-03-27T10:02:33.614 INFO:tasks.ceph.mon.b:Stopped 2023-03-27T10:02:33.615 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-03-27T10:02:33.615 INFO:teuthology.orchestra.run:waiting for 300 2023-03-27T10:02:33.667 INFO:tasks.ceph.mon.c:Stopped 2023-03-27T10:02:33.667 INFO:tasks.ceph:Checking cluster log for badness... 2023-03-27T10:02:33.668 DEBUG:teuthology.orchestra.run.smithi023:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v 'overall HEALTH_' | egrep -v '\(MON_DOWN\)' | head -n 1 2023-03-27T10:02:33.757 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi023.front.sepia.ceph.com 2023-03-27T10:02:33.758 DEBUG:teuthology.orchestra.run.smithi023:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-03-27T10:02:34.639 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi023.front.sepia.ceph.com 2023-03-27T10:02:34.640 DEBUG:teuthology.orchestra.run.smithi023:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-03-27T10:02:34.865 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi023.front.sepia.ceph.com 2023-03-27T10:02:34.866 DEBUG:teuthology.orchestra.run.smithi023:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-03-27T10:02:35.058 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi023.front.sepia.ceph.com 2023-03-27T10:02:35.058 DEBUG:teuthology.orchestra.run.smithi023:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-03-27T10:02:35.301 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi112.front.sepia.ceph.com 2023-03-27T10:02:35.302 DEBUG:teuthology.orchestra.run.smithi112:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-03-27T10:02:35.609 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi112.front.sepia.ceph.com 2023-03-27T10:02:35.610 DEBUG:teuthology.orchestra.run.smithi112:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-03-27T10:02:35.739 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi112.front.sepia.ceph.com 2023-03-27T10:02:35.740 DEBUG:teuthology.orchestra.run.smithi112:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-03-27T10:02:35.830 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi112.front.sepia.ceph.com 2023-03-27T10:02:35.831 DEBUG:teuthology.orchestra.run.smithi112:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-03-27T10:02:35.896 INFO:tasks.ceph:Archiving mon data... 2023-03-27T10:02:35.898 DEBUG:teuthology.misc:Transferring archived files from smithi023:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2023-03-27_06:39:30-fs-wip-rishabh-cephman-6-testing-default-smithi/7221293/data/mon.a.tgz 2023-03-27T10:02:35.899 DEBUG:teuthology.orchestra.run.smithi023:> mktemp 2023-03-27T10:02:35.939 INFO:teuthology.orchestra.run.smithi023.stdout:/tmp/tmp.8IY4FciOql 2023-03-27T10:02:35.940 DEBUG:teuthology.orchestra.run.smithi023:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.8IY4FciOql 2023-03-27T10:02:36.093 DEBUG:teuthology.orchestra.run.smithi023:> sudo chmod 0666 /tmp/tmp.8IY4FciOql 2023-03-27T10:02:36.195 DEBUG:teuthology.orchestra.remote:smithi023:/tmp/tmp.8IY4FciOql is 228KB 2023-03-27T10:02:36.284 DEBUG:teuthology.orchestra.run.smithi023:> rm -fr /tmp/tmp.8IY4FciOql 2023-03-27T10:02:36.321 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2023-03-27_06:39:30-fs-wip-rishabh-cephman-6-testing-default-smithi/7221293/data/mon.b.tgz 2023-03-27T10:02:36.322 DEBUG:teuthology.orchestra.run.smithi112:> mktemp 2023-03-27T10:02:36.343 INFO:teuthology.orchestra.run.smithi112.stdout:/tmp/tmp.KNJvxbbWim 2023-03-27T10:02:36.343 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.KNJvxbbWim 2023-03-27T10:02:36.478 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0666 /tmp/tmp.KNJvxbbWim 2023-03-27T10:02:36.566 DEBUG:teuthology.orchestra.remote:smithi112:/tmp/tmp.KNJvxbbWim is 317KB 2023-03-27T10:02:36.633 DEBUG:teuthology.orchestra.run.smithi112:> rm -fr /tmp/tmp.KNJvxbbWim 2023-03-27T10:02:36.650 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2023-03-27_06:39:30-fs-wip-rishabh-cephman-6-testing-default-smithi/7221293/data/mon.c.tgz 2023-03-27T10:02:36.651 DEBUG:teuthology.orchestra.run.smithi112:> mktemp 2023-03-27T10:02:36.707 INFO:teuthology.orchestra.run.smithi112.stdout:/tmp/tmp.SoFkIf5yeT 2023-03-27T10:02:36.708 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.SoFkIf5yeT 2023-03-27T10:02:36.824 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0666 /tmp/tmp.SoFkIf5yeT 2023-03-27T10:02:36.910 DEBUG:teuthology.orchestra.remote:smithi112:/tmp/tmp.SoFkIf5yeT is 317KB 2023-03-27T10:02:37.005 DEBUG:teuthology.orchestra.run.smithi112:> rm -fr /tmp/tmp.SoFkIf5yeT 2023-03-27T10:02:37.021 INFO:tasks.ceph:Cleaning ceph cluster... 2023-03-27T10:02:37.022 DEBUG:teuthology.orchestra.run.smithi023:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-03-27T10:02:37.026 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-03-27T10:02:37.086 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-03-27T10:02:37.087 DEBUG:teuthology.orchestra.run.smithi023:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-03-27T10:02:37.091 DEBUG:teuthology.orchestra.run.smithi112:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-03-27T10:02:37.160 INFO:teuthology.orchestra.run.smithi023.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-03-27T10:02:37.163 INFO:teuthology.orchestra.run.smithi112.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-03-27T10:02:37.174 INFO:tasks.ceph:Archiving crash dumps... 2023-03-27T10:02:37.177 DEBUG:teuthology.misc:Transferring archived files from smithi023:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-03-27_06:39:30-fs-wip-rishabh-cephman-6-testing-default-smithi/7221293/remote/smithi023/crash 2023-03-27T10:02:37.181 DEBUG:teuthology.orchestra.run.smithi023:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2023-03-27T10:02:37.247 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-03-27_06:39:30-fs-wip-rishabh-cephman-6-testing-default-smithi/7221293/remote/smithi112/crash 2023-03-27T10:02:37.249 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2023-03-27T10:02:37.277 INFO:tasks.ceph:Shutting down logrotate 2023-03-27T10:02:37.277 DEBUG:teuthology.orchestra.run.smithi023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-27T10:02:37.282 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-27T10:02:37.364 DEBUG:teuthology.orchestra.run.smithi023:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-03-27T10:02:37.426 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-03-27T10:02:37.453 INFO:tasks.ceph:Compressing logs... 2023-03-27T10:02:37.455 DEBUG:teuthology.orchestra.run.smithi023:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-03-27T10:02:37.461 DEBUG:teuthology.orchestra.run.smithi112:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-03-27T10:02:40.202 INFO:tasks.ceph:Archiving logs... 2023-03-27T10:02:40.203 DEBUG:teuthology.misc:Transferring archived files from smithi023:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-03-27_06:39:30-fs-wip-rishabh-cephman-6-testing-default-smithi/7221293/remote/smithi023/log 2023-03-27T10:02:40.204 DEBUG:teuthology.orchestra.run.smithi023:> sudo tar cz -f - -C /var/log/ceph -- . 2023-03-27T10:02:40.777 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-03-27_06:39:30-fs-wip-rishabh-cephman-6-testing-default-smithi/7221293/remote/smithi112/log 2023-03-27T10:02:40.779 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar cz -f - -C /var/log/ceph -- . 2023-03-27T10:02:41.272 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa/tasks/cephfs/mount.py", line 192, in is_blocked output = self.fs.get_ceph_cmd_stdout('osd blocklist ls') File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa/tasks/cephfs/filesystem.py", line 248, in get_ceph_cmd_stdout return self.mon_manager.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa/tasks/ceph_manager.py", line 1579, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi023 with status 22: "sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph 'osd blocklist ls'" During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa/tasks/ceph.py", line 1930, in task with contextutil.nested(*subtasks): File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/contextutil.py", line 55, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa/tasks/ceph.py", line 251, in ceph_log yield File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/contextutil.py", line 31, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa/tasks/ceph.py", line 468, in cephfs_setup fs = Filesystem(ctx, fs_config=temp, name=name, create=True) File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa/tasks/cephfs/filesystem.py", line 538, in __init__ self.create() File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa/tasks/cephfs/filesystem.py", line 723, in create self.run_client_payload(f"setfattr -n ceph.dir.layout.pool -v {ec_data_pool_name} . && getfattr -n ceph.dir.layout .") File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa/tasks/cephfs/filesystem.py", line 798, in run_client_payload m.umount_wait(require_clean=True) File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa/tasks/cephfs/fuse_mount.py", line 389, in umount_wait if self.is_blocked(): File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa/tasks/cephfs/mount.py", line 195, in is_blocked output = self.fs.get_ceph_cmd_stdout('osd blacklist ls') File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa/tasks/cephfs/filesystem.py", line 248, in get_ceph_cmd_stdout return self.mon_manager.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa/tasks/ceph_manager.py", line 1579, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi023 with status 22: "sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph 'osd blacklist ls'" 2023-03-27T10:02:41.358 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=5a63601531194be1ae9e737281be9fe6 Traceback (most recent call last): File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa/tasks/cephfs/mount.py", line 192, in is_blocked output = self.fs.get_ceph_cmd_stdout('osd blocklist ls') File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa/tasks/cephfs/filesystem.py", line 248, in get_ceph_cmd_stdout return self.mon_manager.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa/tasks/ceph_manager.py", line 1579, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi023 with status 22: "sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph 'osd blocklist ls'" During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa/tasks/ceph.py", line 1930, in task with contextutil.nested(*subtasks): File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/contextutil.py", line 55, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa/tasks/ceph.py", line 251, in ceph_log yield File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/contextutil.py", line 31, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa/tasks/ceph.py", line 468, in cephfs_setup fs = Filesystem(ctx, fs_config=temp, name=name, create=True) File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa/tasks/cephfs/filesystem.py", line 538, in __init__ self.create() File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa/tasks/cephfs/filesystem.py", line 723, in create self.run_client_payload(f"setfattr -n ceph.dir.layout.pool -v {ec_data_pool_name} . && getfattr -n ceph.dir.layout .") File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa/tasks/cephfs/filesystem.py", line 798, in run_client_payload m.umount_wait(require_clean=True) File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa/tasks/cephfs/fuse_mount.py", line 389, in umount_wait if self.is_blocked(): File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa/tasks/cephfs/mount.py", line 195, in is_blocked output = self.fs.get_ceph_cmd_stdout('osd blacklist ls') File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa/tasks/cephfs/filesystem.py", line 248, in get_ceph_cmd_stdout return self.mon_manager.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_342392241eddd9827618f3dbb8134df7441718a8/qa/tasks/ceph_manager.py", line 1579, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi023 with status 22: "sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph 'osd blacklist ls'" 2023-03-27T10:02:41.366 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-03-27T10:02:41.374 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-03-27T10:02:41.388 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits... 2023-03-27T10:02:41.388 DEBUG:teuthology.orchestra.run.smithi023:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2023-03-27T10:02:41.402 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2023-03-27T10:02:41.515 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-03-27T10:02:41.524 INFO:teuthology.task.clock:Checking final clock skew... 2023-03-27T10:02:41.524 DEBUG:teuthology.orchestra.run.smithi023:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-03-27T10:02:41.529 DEBUG:teuthology.orchestra.run.smithi112:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-03-27T10:02:41.544 INFO:teuthology.orchestra.run.smithi112.stderr:bash: ntpq: command not found 2023-03-27T10:02:41.548 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-03-27T10:02:41.548 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2023-03-27T10:02:41.548 INFO:teuthology.orchestra.run.smithi112.stdout:^+ hv01.front.sepia.ceph.com 2 6 177 38 +629us[ +629us] +/- 89ms 2023-03-27T10:02:41.549 INFO:teuthology.orchestra.run.smithi112.stdout:^* hv02.front.sepia.ceph.com 3 6 177 39 +1659us[+1615us] +/- 74ms 2023-03-27T10:02:41.549 INFO:teuthology.orchestra.run.smithi112.stdout:^+ hv03.front.sepia.ceph.com 3 6 177 38 -2498us[-2498us] +/- 49ms 2023-03-27T10:02:41.549 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-03-27T10:02:41.562 INFO:teuthology.orchestra.run.smithi023.stderr:bash: ntpq: command not found 2023-03-27T10:02:41.569 INFO:teuthology.orchestra.run.smithi023.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-03-27T10:02:41.570 INFO:teuthology.orchestra.run.smithi023.stdout:=============================================================================== 2023-03-27T10:02:41.570 INFO:teuthology.orchestra.run.smithi023.stdout:^+ hv01.front.sepia.ceph.com 2 6 177 39 +1102us[ +622us] +/- 89ms 2023-03-27T10:02:41.570 INFO:teuthology.orchestra.run.smithi023.stdout:^* hv02.front.sepia.ceph.com 3 6 177 38 +2131us[+1650us] +/- 74ms 2023-03-27T10:02:41.570 INFO:teuthology.orchestra.run.smithi023.stdout:^+ hv03.front.sepia.ceph.com 3 6 177 37 -2030us[-2030us] +/- 49ms 2023-03-27T10:02:41.571 INFO:teuthology.orchestra.run.smithi023.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-03-27T10:02:41.571 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-03-27T10:02:41.583 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-03-27T10:02:41.584 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-03-27T10:02:41.595 DEBUG:teuthology.orchestra.run.smithi023:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-03-27T10:02:41.767 DEBUG:teuthology.orchestra.run.smithi112:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-03-27T10:02:41.853 DEBUG:teuthology.orchestra.run.smithi023:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-03-27T10:02:41.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-27T10:02:41.921 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-03-27T10:02:41.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-27T10:02:41.954 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-03-27T10:02:41.968 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-03-27T10:02:41.986 INFO:teuthology.task.internal:Duration was 834.889161 seconds 2023-03-27T10:02:41.987 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-03-27T10:02:41.997 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-03-27T10:02:41.998 DEBUG:teuthology.orchestra.run.smithi023:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-03-27T10:02:42.003 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-03-27T10:02:42.048 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-03-27T10:02:42.104 INFO:teuthology.orchestra.run.smithi023.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-03-27T10:02:42.574 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-03-27T10:02:42.575 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi023.front.sepia.ceph.com 2023-03-27T10:02:42.576 DEBUG:teuthology.orchestra.run.smithi023:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-03-27T10:02:42.629 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi112.front.sepia.ceph.com 2023-03-27T10:02:42.630 DEBUG:teuthology.orchestra.run.smithi112:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-03-27T10:02:42.654 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-03-27T10:02:42.655 DEBUG:teuthology.orchestra.run.smithi023:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-03-27T10:02:42.659 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-03-27T10:02:42.736 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-03-27T10:02:42.737 DEBUG:teuthology.orchestra.run.smithi023:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-03-27T10:02:42.741 DEBUG:teuthology.orchestra.run.smithi112:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-03-27T10:02:42.849 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-03-27T10:02:42.861 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-03-27T10:02:42.862 DEBUG:teuthology.orchestra.run.smithi023:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-03-27T10:02:42.924 DEBUG:teuthology.orchestra.run.smithi112:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-03-27T10:02:42.951 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-03-27T10:02:42.960 DEBUG:teuthology.orchestra.run.smithi023:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-03-27T10:02:42.965 DEBUG:teuthology.orchestra.run.smithi112:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-03-27T10:02:43.016 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = core 2023-03-27T10:02:43.021 INFO:teuthology.orchestra.run.smithi023.stdout:kernel.core_pattern = core 2023-03-27T10:02:43.058 DEBUG:teuthology.orchestra.run.smithi023:> test -e /home/ubuntu/cephtest/archive/coredump 2023-03-27T10:02:43.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-27T10:02:43.097 DEBUG:teuthology.orchestra.run.smithi112:> test -e /home/ubuntu/cephtest/archive/coredump 2023-03-27T10:02:43.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-27T10:02:43.115 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-03-27T10:02:43.128 INFO:teuthology.task.internal:Transferring archived files... 2023-03-27T10:02:43.130 DEBUG:teuthology.misc:Transferring archived files from smithi023:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-03-27_06:39:30-fs-wip-rishabh-cephman-6-testing-default-smithi/7221293/remote/smithi023 2023-03-27T10:02:43.130 DEBUG:teuthology.orchestra.run.smithi023:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2023-03-27T10:02:43.223 DEBUG:teuthology.misc:Transferring archived files from smithi112:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-03-27_06:39:30-fs-wip-rishabh-cephman-6-testing-default-smithi/7221293/remote/smithi112 2023-03-27T10:02:43.224 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2023-03-27T10:02:43.273 INFO:teuthology.task.internal:Removing archive directory... 2023-03-27T10:02:43.274 DEBUG:teuthology.orchestra.run.smithi023:> rm -rf -- /home/ubuntu/cephtest/archive 2023-03-27T10:02:43.279 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf -- /home/ubuntu/cephtest/archive 2023-03-27T10:02:43.328 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-03-27T10:02:43.340 INFO:teuthology.task.internal:Not uploading archives. 2023-03-27T10:02:43.341 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-03-27T10:02:43.359 INFO:teuthology.task.internal:Tidying up after the test... 2023-03-27T10:02:43.362 DEBUG:teuthology.orchestra.run.smithi023:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-03-27T10:02:43.374 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-03-27T10:02:43.398 INFO:teuthology.orchestra.run.smithi112.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 27 10:02 /home/ubuntu/cephtest 2023-03-27T21:45:38.300 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2023-03-27T21:45:38.356 DEBUG:teuthology.task.console_log:Killing console logger for smithi023 2023-03-27T21:45:38.357 DEBUG:teuthology.task.console_log:Killing console logger for smithi112 2023-03-27T21:45:38.358 DEBUG:teuthology.exit:Finished running handlers