2023-06-23T00:50:48.985 INFO:root:teuthology version: 0.0.1.dev128+g076bbeb 2023-06-23T00:50:48.986 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_yuriw@teuthology --name yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi --archive /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311926 --verbose --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/no standby-replay subvolume/{with-namespace-isolated} tasks/{0-check-counter workunit/suites/fsstress} wsync/{no}} -- /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311926/orig.config.yaml 2023-06-23T00:50:49.043 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-23T00:50:49.145 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311926 branch: wip-yuri4-testing-2023-06-21-0846-quincy description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/no standby-replay subvolume/{with-namespace-isolated} tasks/{0-check-counter workunit/suites/fsstress} wsync/{no}} email: yweinste@redhat.com first_in_suite: false job_id: '7311926' kernel: client: branch: testing kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.4' overrides: admin_socket: branch: wip-yuri4-testing-2023-06-21-0846-quincy ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 42ddc46e6e6582d8e29a5a982a81886fe028be2c subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 42ddc46e6e6582d8e29a5a982a81886fe028be2c kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - nowsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri4-testing-2023-06-21-0846-quincy sha1: 42ddc46e6e6582d8e29a5a982a81886fe028be2c owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 9037 sha1: 42ddc46e6e6582d8e29a5a982a81886fe028be2c sleep_before_teardown: 0 subset: 111/120000 suite: fs suite_branch: wip-yuri4-testing-2023-06-21-0846-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 42ddc46e6e6582d8e29a5a982a81886fe028be2c targets: smithi120.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ6Gw4c6Wuh55BSbPgPQzg/VvHfenNu8gJCtRHYvUY/SLgFHx7pJ5QOw8bw781Efgu+44roU0rYFUEoea+siels= smithi134.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCqbMNWpk8JK7EEWwa+i86XieFoQ7Oz6383Of1HxWdKzciwj65oZ3grvieXdPsQrjNXLSskieDgL+TrdgNVRy5kSxiX21olKG6XTgbxWbw1/6NsLbnXMMbKPmzjrlDKiTPhpbN9myR4DZyMBwSJlA8oYqnXdBWEcxxNV7M0NT6YBz5P2WS2/r4N6bvAnojjMc95dGH6pq40lg1PDn2qPwd+mEJolkRUr81sdxJhnQCADb54k6R8Wi6nVokYn5UIOPMoKCXJfx1cMxWIugivkdK2n1XUPDdyY7Dk1tU9rifx4QPgnB5FhH41KERhc48ooxGYTln2P8YlPPhHnsEdRrrFuc2CBFQZJ4XvcT3hEd328GxILbSKHlpUpaRdrYqwCf57lJGBRksYLdyYCNxrMf83RjwY0Z2BkCQG1Gr4sOJukwqD4rDqjTBe56nYDSq8bS3qIuCIN7dlhb13G5cY/NzJCQBvzFkj2cazTOHb0pqhJHaoTxaLr+Wbi6R9xr2nxlU= smithi157.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMFEBtPcrY8XOJPHcTEL+TCv+28K01iw+hW5n3tKkPWTQ33e/kONGkSJPKr06gZsfmpiRvqSGEQASzbmBk5Nztg= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - workunit: clients: all: - suites/fsstress.sh timeout: 6h teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 076bbebc42a14f7d568aaa78eabb0038327bcb23 timestamp: 2023-06-22_15:25:25 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-06-23T00:50:49.145 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa; will attempt to use it 2023-06-23T00:50:49.146 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks 2023-06-23T00:50:49.147 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-06-23T00:50:49.151 INFO:teuthology.task.internal:Checking packages... 2023-06-23T00:50:49.190 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '42ddc46e6e6582d8e29a5a982a81886fe028be2c' 2023-06-23T00:50:49.190 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-06-23T00:50:49.191 INFO:teuthology.packaging:ref: None 2023-06-23T00:50:49.191 INFO:teuthology.packaging:tag: None 2023-06-23T00:50:49.191 INFO:teuthology.packaging:branch: wip-yuri4-testing-2023-06-21-0846-quincy 2023-06-23T00:50:49.191 INFO:teuthology.packaging:sha1: 42ddc46e6e6582d8e29a5a982a81886fe028be2c 2023-06-23T00:50:49.192 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri4-testing-2023-06-21-0846-quincy 2023-06-23T00:50:49.339 INFO:teuthology.task.internal:Found packages for ceph version 17.2.6-719.g42ddc46e 2023-06-23T00:50:49.339 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-06-23T00:50:49.356 INFO:teuthology.task.internal:no buildpackages task found 2023-06-23T00:50:49.356 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-06-23T00:50:49.393 INFO:teuthology.task.internal:Saving configuration 2023-06-23T00:50:49.415 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-06-23T00:50:49.441 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-23T00:50:49.470 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi120.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311926', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-23 00:45:09.815243', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ6Gw4c6Wuh55BSbPgPQzg/VvHfenNu8gJCtRHYvUY/SLgFHx7pJ5QOw8bw781Efgu+44roU0rYFUEoea+siels='} 2023-06-23T00:50:49.496 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi134.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311926', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-23 00:45:09.817576', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFktpzMtcMWML45uj5xyWU1UEh9mllR2w+ucUORMpjGzQ1xh5+JICj9oEWzg06pr9fukTY9mJZxnVi3uBG2l+iE='} 2023-06-23T00:50:49.523 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311926', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-23 00:45:09.819354', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMFEBtPcrY8XOJPHcTEL+TCv+28K01iw+hW5n3tKkPWTQ33e/kONGkSJPKr06gZsfmpiRvqSGEQASzbmBk5Nztg='} 2023-06-23T00:50:49.524 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-06-23T00:50:49.533 INFO:teuthology.task.internal:roles: ubuntu@smithi120.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-06-23T00:50:49.534 INFO:teuthology.task.internal:roles: ubuntu@smithi134.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-06-23T00:50:49.534 INFO:teuthology.task.internal:roles: ubuntu@smithi157.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-06-23T00:50:49.534 INFO:teuthology.run_tasks:Running task console_log... 2023-06-23T00:50:49.692 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9deeadd0d0>, signals=[15]) 2023-06-23T00:50:49.693 INFO:teuthology.run_tasks:Running task internal.connect... 2023-06-23T00:50:49.711 INFO:teuthology.task.internal:Opening connections... 2023-06-23T00:50:49.712 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi120.front.sepia.ceph.com 2023-06-23T00:50:49.717 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T00:50:49.788 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi134.front.sepia.ceph.com 2023-06-23T00:50:49.789 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T00:50:49.899 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi157.front.sepia.ceph.com 2023-06-23T00:50:49.900 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T00:50:49.969 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-06-23T00:50:49.983 DEBUG:teuthology.orchestra.run.smithi120:> uname -m 2023-06-23T00:50:50.000 INFO:teuthology.orchestra.run.smithi120.stdout:x86_64 2023-06-23T00:50:50.001 DEBUG:teuthology.orchestra.run.smithi120:> cat /etc/os-release 2023-06-23T00:50:50.056 INFO:teuthology.orchestra.run.smithi120.stdout:NAME="Red Hat Enterprise Linux" 2023-06-23T00:50:50.057 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION="8.4 (Ootpa)" 2023-06-23T00:50:50.057 INFO:teuthology.orchestra.run.smithi120.stdout:ID="rhel" 2023-06-23T00:50:50.057 INFO:teuthology.orchestra.run.smithi120.stdout:ID_LIKE="fedora" 2023-06-23T00:50:50.057 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION_ID="8.4" 2023-06-23T00:50:50.058 INFO:teuthology.orchestra.run.smithi120.stdout:PLATFORM_ID="platform:el8" 2023-06-23T00:50:50.058 INFO:teuthology.orchestra.run.smithi120.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-06-23T00:50:50.058 INFO:teuthology.orchestra.run.smithi120.stdout:ANSI_COLOR="0;31" 2023-06-23T00:50:50.058 INFO:teuthology.orchestra.run.smithi120.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-06-23T00:50:50.059 INFO:teuthology.orchestra.run.smithi120.stdout:HOME_URL="https://www.redhat.com/" 2023-06-23T00:50:50.059 INFO:teuthology.orchestra.run.smithi120.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-23T00:50:50.059 INFO:teuthology.orchestra.run.smithi120.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-23T00:50:50.059 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T00:50:50.060 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-23T00:50:50.060 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-06-23T00:50:50.060 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-23T00:50:50.061 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-06-23T00:50:50.062 INFO:teuthology.lock.ops:Updating smithi120.front.sepia.ceph.com on lock server 2023-06-23T00:50:50.093 DEBUG:teuthology.orchestra.run.smithi134:> uname -m 2023-06-23T00:50:50.110 INFO:teuthology.orchestra.run.smithi134.stdout:x86_64 2023-06-23T00:50:50.111 DEBUG:teuthology.orchestra.run.smithi134:> cat /etc/os-release 2023-06-23T00:50:50.167 INFO:teuthology.orchestra.run.smithi134.stdout:NAME="Red Hat Enterprise Linux" 2023-06-23T00:50:50.168 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION="8.4 (Ootpa)" 2023-06-23T00:50:50.168 INFO:teuthology.orchestra.run.smithi134.stdout:ID="rhel" 2023-06-23T00:50:50.168 INFO:teuthology.orchestra.run.smithi134.stdout:ID_LIKE="fedora" 2023-06-23T00:50:50.168 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION_ID="8.4" 2023-06-23T00:50:50.168 INFO:teuthology.orchestra.run.smithi134.stdout:PLATFORM_ID="platform:el8" 2023-06-23T00:50:50.169 INFO:teuthology.orchestra.run.smithi134.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-06-23T00:50:50.169 INFO:teuthology.orchestra.run.smithi134.stdout:ANSI_COLOR="0;31" 2023-06-23T00:50:50.169 INFO:teuthology.orchestra.run.smithi134.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-06-23T00:50:50.169 INFO:teuthology.orchestra.run.smithi134.stdout:HOME_URL="https://www.redhat.com/" 2023-06-23T00:50:50.169 INFO:teuthology.orchestra.run.smithi134.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-23T00:50:50.169 INFO:teuthology.orchestra.run.smithi134.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-23T00:50:50.169 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-06-23T00:50:50.170 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-23T00:50:50.170 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-06-23T00:50:50.170 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-23T00:50:50.170 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-06-23T00:50:50.170 INFO:teuthology.lock.ops:Updating smithi134.front.sepia.ceph.com on lock server 2023-06-23T00:50:50.193 DEBUG:teuthology.orchestra.run.smithi157:> uname -m 2023-06-23T00:50:50.210 INFO:teuthology.orchestra.run.smithi157.stdout:x86_64 2023-06-23T00:50:50.211 DEBUG:teuthology.orchestra.run.smithi157:> cat /etc/os-release 2023-06-23T00:50:50.266 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="Red Hat Enterprise Linux" 2023-06-23T00:50:50.267 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="8.4 (Ootpa)" 2023-06-23T00:50:50.267 INFO:teuthology.orchestra.run.smithi157.stdout:ID="rhel" 2023-06-23T00:50:50.267 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE="fedora" 2023-06-23T00:50:50.267 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="8.4" 2023-06-23T00:50:50.267 INFO:teuthology.orchestra.run.smithi157.stdout:PLATFORM_ID="platform:el8" 2023-06-23T00:50:50.267 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-06-23T00:50:50.268 INFO:teuthology.orchestra.run.smithi157.stdout:ANSI_COLOR="0;31" 2023-06-23T00:50:50.268 INFO:teuthology.orchestra.run.smithi157.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-06-23T00:50:50.268 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://www.redhat.com/" 2023-06-23T00:50:50.268 INFO:teuthology.orchestra.run.smithi157.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-23T00:50:50.269 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-23T00:50:50.269 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-06-23T00:50:50.269 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-23T00:50:50.270 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-06-23T00:50:50.270 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-23T00:50:50.270 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-06-23T00:50:50.271 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server 2023-06-23T00:50:50.293 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-06-23T00:50:50.312 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-06-23T00:50:50.358 INFO:teuthology.task.internal:Checking for old test directory... 2023-06-23T00:50:50.359 DEBUG:teuthology.orchestra.run.smithi120:> test '!' -e /home/ubuntu/cephtest 2023-06-23T00:50:50.362 DEBUG:teuthology.orchestra.run.smithi134:> test '!' -e /home/ubuntu/cephtest 2023-06-23T00:50:50.365 DEBUG:teuthology.orchestra.run.smithi157:> test '!' -e /home/ubuntu/cephtest 2023-06-23T00:50:50.382 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-06-23T00:50:50.387 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-06-23T00:50:50.388 DEBUG:teuthology.orchestra.run.smithi120:> test -z $(ls -A /var/lib/ceph) 2023-06-23T00:50:50.418 DEBUG:teuthology.orchestra.run.smithi134:> test -z $(ls -A /var/lib/ceph) 2023-06-23T00:50:50.421 DEBUG:teuthology.orchestra.run.smithi157:> test -z $(ls -A /var/lib/ceph) 2023-06-23T00:50:50.431 INFO:teuthology.orchestra.run.smithi120.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-06-23T00:50:50.435 INFO:teuthology.orchestra.run.smithi134.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-06-23T00:50:50.439 INFO:teuthology.orchestra.run.smithi157.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-06-23T00:50:50.439 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-06-23T00:50:50.536 INFO:teuthology.run_tasks:Running task kernel... 2023-06-23T00:50:50.561 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2023-06-23T00:50:50.563 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2023-06-23T00:50:50.563 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2023-06-23T00:50:50.564 DEBUG:teuthology.orchestra.run.smithi120:> test -f /run/.containerenv -o -f /.dockerenv 2023-06-23T00:50:50.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T00:50:50.582 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-06-23T00:50:50.827 DEBUG:teuthology.task.kernel:sha1 for client.0 is 1c622d92b6f18ce44a7c33802686ff61c3c273ab 2023-06-23T00:50:50.828 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "1c622d92b6f18ce44a7c33802686ff61c3c273ab"... 2023-06-23T00:50:50.828 DEBUG:teuthology.orchestra.run.smithi120:> uname -r 2023-06-23T00:50:50.844 INFO:teuthology.orchestra.run.smithi120.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-06-23T00:50:50.845 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-305.25.1.el8_4.x86_64 vs 1c622d92b6f18ce44a7c33802686ff61c3c273ab 2023-06-23T00:50:50.845 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-06-23T00:50:50.845 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum upgrade -y linux-firmware 2023-06-23T00:50:52.238 INFO:teuthology.orchestra.run.smithi120.stdout:Updating Subscription Management repositories. 2023-06-23T00:50:52.239 INFO:teuthology.orchestra.run.smithi120.stdout:Unable to read consumer identity 2023-06-23T00:50:55.982 INFO:teuthology.orchestra.run.smithi120.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2023-06-23T00:50:58.168 INFO:teuthology.orchestra.run.smithi120.stdout:Extra Packages for Enterprise Linux 7.3 MB/s | 16 MB 00:02 2023-06-23T00:51:01.461 INFO:teuthology.orchestra.run.smithi120.stdout:lab-extras 4.2 MB/s | 24 kB 00:00 2023-06-23T00:51:02.328 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2023-06-23T00:51:02.329 INFO:teuthology.orchestra.run.smithi120.stdout:Nothing to do. 2023-06-23T00:51:02.329 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2023-06-23T00:51:02.446 INFO:teuthology.task.kernel:Downloading kernel 1c622d92b6f18ce44a7c33802686ff61c3c273ab on client.0... 2023-06-23T00:51:02.447 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1c622d92b6f18ce44a7c33802686ff61c3c273ab 2023-06-23T00:51:02.590 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/1c622d92b6f18ce44a7c33802686ff61c3c273ab/centos/8/flavors/default/x86_64/ 2023-06-23T00:51:02.591 DEBUG:teuthology.orchestra.run.smithi120:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.4.0_rc5_g1c622d92b6f1-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/1c622d92b6f18ce44a7c33802686ff61c3c273ab/centos/8/flavors/default/x86_64/ --input-file=- 2023-06-23T00:51:02.593 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi120.front.sepia.ceph.com to complete... 2023-06-23T00:51:15.568 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23 00:51:15 URL:https://2.chacra.ceph.com/r/kernel/testing/1c622d92b6f18ce44a7c33802686ff61c3c273ab/centos/8/flavors/default/x86_64/kernel-6.4.0_rc5_g1c622d92b6f1-1.x86_64.rpm [379883408/379883408] -> "/tmp/kernel.x86_64.rpm" [1] 2023-06-23T00:51:15.568 INFO:teuthology.orchestra.run.smithi120.stderr:FINISHED --2023-06-23 00:51:15-- 2023-06-23T00:51:15.569 INFO:teuthology.orchestra.run.smithi120.stderr:Total wall clock time: 13s 2023-06-23T00:51:15.569 INFO:teuthology.orchestra.run.smithi120.stderr:Downloaded: 1 files, 362M in 13s (28.6 MB/s) 2023-06-23T00:51:15.570 INFO:teuthology.task.kernel:Installing kernel 1c622d92b6f18ce44a7c33802686ff61c3c273ab on client.0... 2023-06-23T00:51:15.570 DEBUG:teuthology.orchestra.run.smithi120:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-06-23T00:51:17.272 INFO:teuthology.orchestra.run.smithi120.stdout:Verifying... ######################################## 2023-06-23T00:51:17.702 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing... ######################################## 2023-06-23T00:51:17.836 INFO:teuthology.orchestra.run.smithi120.stdout:Updating / installing... 2023-06-23T00:51:55.951 INFO:teuthology.orchestra.run.smithi120.stdout:kernel-6.4.0_rc5_g1c622d92b6f1-1 ######################################## 2023-06-23T00:52:19.494 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi120.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-06-23T00:52:19.495 DEBUG:teuthology.orchestra.run.smithi120:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-06-23T00:52:19.539 INFO:teuthology.orchestra.run.smithi120.stdout:/boot/System.map-6.4.0-rc5-g1c622d92b6f1 2023-06-23T00:52:19.540 INFO:teuthology.orchestra.run.smithi120.stdout:/boot/config-6.4.0-rc5-g1c622d92b6f1 2023-06-23T00:52:19.540 INFO:teuthology.orchestra.run.smithi120.stdout:/boot/vmlinuz-6.4.0-rc5-g1c622d92b6f1 2023-06-23T00:52:19.540 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1 2023-06-23T00:52:19.540 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel 2023-06-23T00:52:19.540 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch 2023-06-23T00:52:19.540 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86 2023-06-23T00:52:19.541 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto 2023-06-23T00:52:19.541 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-06-23T00:52:19.541 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-06-23T00:52:19.541 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-06-23T00:52:19.541 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-06-23T00:52:19.541 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-06-23T00:52:19.541 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-06-23T00:52:19.541 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-06-23T00:52:19.542 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/crc32c-intel.ko 2023-06-23T00:52:19.542 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-06-23T00:52:19.542 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-06-23T00:52:19.542 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-06-23T00:52:19.542 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-06-23T00:52:19.542 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/serpent-avx2.ko 2023-06-23T00:52:19.542 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-06-23T00:52:19.543 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-06-23T00:52:19.543 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-06-23T00:52:19.543 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-06-23T00:52:19.543 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-06-23T00:52:19.543 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-06-23T00:52:19.543 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-06-23T00:52:19.543 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/kernel 2023-06-23T00:52:19.543 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/kernel/cpu 2023-06-23T00:52:19.544 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/kernel/cpu/mce 2023-06-23T00:52:19.544 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-06-23T00:52:19.544 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/kvm 2023-06-23T00:52:19.544 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/kvm/kvm-amd.ko 2023-06-23T00:52:19.544 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/kvm/kvm-intel.ko 2023-06-23T00:52:19.544 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/kvm/kvm.ko 2023-06-23T00:52:19.544 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto 2023-06-23T00:52:19.545 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/ansi_cprng.ko 2023-06-23T00:52:19.545 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/anubis.ko 2023-06-23T00:52:19.545 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/arc4.ko 2023-06-23T00:52:19.545 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/async_tx 2023-06-23T00:52:19.545 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/async_tx/async_memcpy.ko 2023-06-23T00:52:19.545 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/async_tx/async_pq.ko 2023-06-23T00:52:19.545 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/async_tx/async_raid6_recov.ko 2023-06-23T00:52:19.545 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/async_tx/async_tx.ko 2023-06-23T00:52:19.546 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/async_tx/async_xor.ko 2023-06-23T00:52:19.546 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/async_tx/raid6test.ko 2023-06-23T00:52:19.546 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/authenc.ko 2023-06-23T00:52:19.546 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/authencesn.ko 2023-06-23T00:52:19.546 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/blake2b_generic.ko 2023-06-23T00:52:19.546 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/blowfish_common.ko 2023-06-23T00:52:19.546 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/blowfish_generic.ko 2023-06-23T00:52:19.546 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/camellia_generic.ko 2023-06-23T00:52:19.547 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/cast5_generic.ko 2023-06-23T00:52:19.547 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/cast6_generic.ko 2023-06-23T00:52:19.547 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/cast_common.ko 2023-06-23T00:52:19.547 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/ccm.ko 2023-06-23T00:52:19.547 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/cmac.ko 2023-06-23T00:52:19.547 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/crc32_generic.ko 2023-06-23T00:52:19.547 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/crypto_engine.ko 2023-06-23T00:52:19.548 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/crypto_user.ko 2023-06-23T00:52:19.548 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/cts.ko 2023-06-23T00:52:19.548 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/des_generic.ko 2023-06-23T00:52:19.548 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/dh_generic.ko 2023-06-23T00:52:19.548 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/echainiv.ko 2023-06-23T00:52:19.548 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/essiv.ko 2023-06-23T00:52:19.548 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/fcrypt.ko 2023-06-23T00:52:19.548 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/khazad.ko 2023-06-23T00:52:19.549 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/lz4.ko 2023-06-23T00:52:19.549 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/lz4hc.ko 2023-06-23T00:52:19.549 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/md4.ko 2023-06-23T00:52:19.549 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/michael_mic.ko 2023-06-23T00:52:19.549 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/pcbc.ko 2023-06-23T00:52:19.549 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/pcrypt.ko 2023-06-23T00:52:19.549 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/rmd160.ko 2023-06-23T00:52:19.550 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/seed.ko 2023-06-23T00:52:19.550 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/serpent_generic.ko 2023-06-23T00:52:19.550 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/tcrypt.ko 2023-06-23T00:52:19.550 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/tea.ko 2023-06-23T00:52:19.550 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/twofish_common.ko 2023-06-23T00:52:19.550 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/twofish_generic.ko 2023-06-23T00:52:19.550 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/vmac.ko 2023-06-23T00:52:19.550 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/wp512.ko 2023-06-23T00:52:19.551 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/xcbc.ko 2023-06-23T00:52:19.551 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/xor.ko 2023-06-23T00:52:19.551 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/xxhash_generic.ko 2023-06-23T00:52:19.551 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers 2023-06-23T00:52:19.551 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/acpi 2023-06-23T00:52:19.551 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/acpi/acpi_ipmi.ko 2023-06-23T00:52:19.551 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/acpi/acpi_pad.ko 2023-06-23T00:52:19.551 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/acpi/custom_method.ko 2023-06-23T00:52:19.552 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/acpi/ec_sys.ko 2023-06-23T00:52:19.552 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/acpi/platform_profile.ko 2023-06-23T00:52:19.552 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/acpi/sbs.ko 2023-06-23T00:52:19.552 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/acpi/sbshc.ko 2023-06-23T00:52:19.552 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/acpi/video.ko 2023-06-23T00:52:19.552 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata 2023-06-23T00:52:19.552 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/acard-ahci.ko 2023-06-23T00:52:19.553 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/ahci_platform.ko 2023-06-23T00:52:19.553 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/ata_generic.ko 2023-06-23T00:52:19.553 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/libahci_platform.ko 2023-06-23T00:52:19.553 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_acpi.ko 2023-06-23T00:52:19.553 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_ali.ko 2023-06-23T00:52:19.553 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_amd.ko 2023-06-23T00:52:19.554 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_artop.ko 2023-06-23T00:52:19.554 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_atiixp.ko 2023-06-23T00:52:19.554 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_atp867x.ko 2023-06-23T00:52:19.554 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_cmd640.ko 2023-06-23T00:52:19.554 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_cmd64x.ko 2023-06-23T00:52:19.554 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_cypress.ko 2023-06-23T00:52:19.554 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_efar.ko 2023-06-23T00:52:19.554 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_hpt366.ko 2023-06-23T00:52:19.555 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_hpt37x.ko 2023-06-23T00:52:19.555 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_hpt3x2n.ko 2023-06-23T00:52:19.555 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_hpt3x3.ko 2023-06-23T00:52:19.555 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_it8213.ko 2023-06-23T00:52:19.555 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_it821x.ko 2023-06-23T00:52:19.555 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_jmicron.ko 2023-06-23T00:52:19.555 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_marvell.ko 2023-06-23T00:52:19.555 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_mpiix.ko 2023-06-23T00:52:19.556 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_netcell.ko 2023-06-23T00:52:19.556 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_ninja32.ko 2023-06-23T00:52:19.556 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_ns87410.ko 2023-06-23T00:52:19.556 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_ns87415.ko 2023-06-23T00:52:19.557 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_oldpiix.ko 2023-06-23T00:52:19.557 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_opti.ko 2023-06-23T00:52:19.558 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_optidma.ko 2023-06-23T00:52:19.558 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_pcmcia.ko 2023-06-23T00:52:19.558 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_pdc2027x.ko 2023-06-23T00:52:19.558 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-06-23T00:52:19.558 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_piccolo.ko 2023-06-23T00:52:19.558 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_rdc.ko 2023-06-23T00:52:19.559 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_sch.ko 2023-06-23T00:52:19.559 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_serverworks.ko 2023-06-23T00:52:19.559 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_sil680.ko 2023-06-23T00:52:19.559 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_sis.ko 2023-06-23T00:52:19.559 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_sl82c105.ko 2023-06-23T00:52:19.559 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_triflex.ko 2023-06-23T00:52:19.559 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_via.ko 2023-06-23T00:52:19.560 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pdc_adma.ko 2023-06-23T00:52:19.560 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/sata_inic162x.ko 2023-06-23T00:52:19.560 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/sata_mv.ko 2023-06-23T00:52:19.560 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/sata_nv.ko 2023-06-23T00:52:19.560 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/sata_promise.ko 2023-06-23T00:52:19.560 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/sata_qstor.ko 2023-06-23T00:52:19.560 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/sata_sil.ko 2023-06-23T00:52:19.561 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/sata_sil24.ko 2023-06-23T00:52:19.561 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/sata_sis.ko 2023-06-23T00:52:19.561 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/sata_svw.ko 2023-06-23T00:52:19.561 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/sata_sx4.ko 2023-06-23T00:52:19.561 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/sata_uli.ko 2023-06-23T00:52:19.561 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/sata_via.ko 2023-06-23T00:52:19.561 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/sata_vsc.ko 2023-06-23T00:52:19.561 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/auxdisplay 2023-06-23T00:52:19.562 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/auxdisplay/ks0108.ko 2023-06-23T00:52:19.562 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/base 2023-06-23T00:52:19.562 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/base/regmap 2023-06-23T00:52:19.562 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/base/regmap/regmap-i2c.ko 2023-06-23T00:52:19.562 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/bcma 2023-06-23T00:52:19.562 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/bcma/bcma.ko 2023-06-23T00:52:19.562 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block 2023-06-23T00:52:19.562 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/aoe 2023-06-23T00:52:19.563 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/aoe/aoe.ko 2023-06-23T00:52:19.563 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/brd.ko 2023-06-23T00:52:19.563 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/drbd 2023-06-23T00:52:19.563 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/drbd/drbd.ko 2023-06-23T00:52:19.563 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/floppy.ko 2023-06-23T00:52:19.563 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/loop.ko 2023-06-23T00:52:19.563 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/mtip32xx 2023-06-23T00:52:19.564 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-06-23T00:52:19.564 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/nbd.ko 2023-06-23T00:52:19.564 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/null_blk 2023-06-23T00:52:19.564 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/null_blk/null_blk.ko 2023-06-23T00:52:19.564 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/pktcdvd.ko 2023-06-23T00:52:19.564 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/rbd.ko 2023-06-23T00:52:19.564 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/virtio_blk.ko 2023-06-23T00:52:19.564 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/xen-blkback 2023-06-23T00:52:19.565 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-06-23T00:52:19.565 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/xen-blkfront.ko 2023-06-23T00:52:19.565 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/zram 2023-06-23T00:52:19.565 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/zram/zram.ko 2023-06-23T00:52:19.565 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char 2023-06-23T00:52:19.565 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/hangcheck-timer.ko 2023-06-23T00:52:19.565 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/hw_random 2023-06-23T00:52:19.566 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/hw_random/amd-rng.ko 2023-06-23T00:52:19.566 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/hw_random/intel-rng.ko 2023-06-23T00:52:19.566 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-06-23T00:52:19.566 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/hw_random/via-rng.ko 2023-06-23T00:52:19.566 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/hw_random/virtio-rng.ko 2023-06-23T00:52:19.566 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/ipmi 2023-06-23T00:52:19.566 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-06-23T00:52:19.566 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-06-23T00:52:19.567 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-06-23T00:52:19.567 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/ipmi/ipmi_si.ko 2023-06-23T00:52:19.567 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-06-23T00:52:19.567 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-06-23T00:52:19.567 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/lp.ko 2023-06-23T00:52:19.567 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/mwave 2023-06-23T00:52:19.567 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/mwave/mwave.ko 2023-06-23T00:52:19.567 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/ppdev.ko 2023-06-23T00:52:19.568 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/tlclk.ko 2023-06-23T00:52:19.568 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/tpm 2023-06-23T00:52:19.568 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/tpm/tpm.ko 2023-06-23T00:52:19.568 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/tpm/tpm_atmel.ko 2023-06-23T00:52:19.568 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/tpm/tpm_infineon.ko 2023-06-23T00:52:19.568 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/tpm/tpm_nsc.ko 2023-06-23T00:52:19.568 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/tpm/tpm_tis.ko 2023-06-23T00:52:19.569 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-06-23T00:52:19.569 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/uv_mmtimer.ko 2023-06-23T00:52:19.569 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/virtio_console.ko 2023-06-23T00:52:19.569 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/cpufreq 2023-06-23T00:52:19.569 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-06-23T00:52:19.569 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-06-23T00:52:19.569 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/cpufreq/p4-clockmod.ko 2023-06-23T00:52:19.569 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-06-23T00:52:19.570 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/cpufreq/powernow-k8.ko 2023-06-23T00:52:19.570 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/cpufreq/speedstep-lib.ko 2023-06-23T00:52:19.570 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/crypto 2023-06-23T00:52:19.570 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/crypto/ccp 2023-06-23T00:52:19.570 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-06-23T00:52:19.570 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/crypto/ccp/ccp.ko 2023-06-23T00:52:19.570 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/crypto/intel 2023-06-23T00:52:19.570 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/crypto/intel/qat 2023-06-23T00:52:19.571 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/crypto/intel/qat/qat_common 2023-06-23T00:52:19.571 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-06-23T00:52:19.571 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-06-23T00:52:19.571 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-06-23T00:52:19.571 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/crypto/padlock-aes.ko 2023-06-23T00:52:19.571 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/crypto/padlock-sha.ko 2023-06-23T00:52:19.571 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/crypto/virtio 2023-06-23T00:52:19.571 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-06-23T00:52:19.572 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/dca 2023-06-23T00:52:19.572 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/dca/dca.ko 2023-06-23T00:52:19.572 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/devfreq 2023-06-23T00:52:19.572 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-06-23T00:52:19.572 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/dma 2023-06-23T00:52:19.572 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/dma/dw 2023-06-23T00:52:19.572 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/dma/dw/dw_dmac.ko 2023-06-23T00:52:19.573 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/dma/ioat 2023-06-23T00:52:19.573 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/dma/ioat/ioatdma.ko 2023-06-23T00:52:19.573 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/edac 2023-06-23T00:52:19.573 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/edac/amd64_edac.ko 2023-06-23T00:52:19.573 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/edac/e752x_edac.ko 2023-06-23T00:52:19.573 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/edac/edac_mce_amd.ko 2023-06-23T00:52:19.573 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/edac/i3000_edac.ko 2023-06-23T00:52:19.573 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/edac/i3200_edac.ko 2023-06-23T00:52:19.574 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/edac/i5100_edac.ko 2023-06-23T00:52:19.574 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/edac/i5400_edac.ko 2023-06-23T00:52:19.574 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/edac/i7300_edac.ko 2023-06-23T00:52:19.574 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/edac/i7core_edac.ko 2023-06-23T00:52:19.574 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/edac/i82975x_edac.ko 2023-06-23T00:52:19.574 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/edac/ie31200_edac.ko 2023-06-23T00:52:19.574 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/edac/sb_edac.ko 2023-06-23T00:52:19.574 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/edac/x38_edac.ko 2023-06-23T00:52:19.575 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/firmware 2023-06-23T00:52:19.575 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/firmware/edd.ko 2023-06-23T00:52:19.575 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/firmware/iscsi_ibft.ko 2023-06-23T00:52:19.575 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/gpio 2023-06-23T00:52:19.575 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/gpio/gpio-ich.ko 2023-06-23T00:52:19.575 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/gpio/gpio-viperboard.ko 2023-06-23T00:52:19.575 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid 2023-06-23T00:52:19.575 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-appleir.ko 2023-06-23T00:52:19.576 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-aureal.ko 2023-06-23T00:52:19.576 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-axff.ko 2023-06-23T00:52:19.576 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-dr.ko 2023-06-23T00:52:19.576 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-elecom.ko 2023-06-23T00:52:19.576 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-elo.ko 2023-06-23T00:52:19.576 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-emsff.ko 2023-06-23T00:52:19.576 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-gaff.ko 2023-06-23T00:52:19.577 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-gt683r.ko 2023-06-23T00:52:19.577 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-gyration.ko 2023-06-23T00:52:19.577 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-holtek-kbd.ko 2023-06-23T00:52:19.577 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-holtek-mouse.ko 2023-06-23T00:52:19.577 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-holtekff.ko 2023-06-23T00:52:19.577 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-hyperv.ko 2023-06-23T00:52:19.577 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-icade.ko 2023-06-23T00:52:19.577 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-keytouch.ko 2023-06-23T00:52:19.578 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-kye.ko 2023-06-23T00:52:19.578 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-lcpower.ko 2023-06-23T00:52:19.578 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-led.ko 2023-06-23T00:52:19.578 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-lenovo.ko 2023-06-23T00:52:19.578 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-logitech-dj.ko 2023-06-23T00:52:19.578 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-06-23T00:52:19.578 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-multitouch.ko 2023-06-23T00:52:19.578 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-ortek.ko 2023-06-23T00:52:19.579 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-penmount.ko 2023-06-23T00:52:19.579 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-petalynx.ko 2023-06-23T00:52:19.579 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-picolcd.ko 2023-06-23T00:52:19.579 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-pl.ko 2023-06-23T00:52:19.579 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-plantronics.ko 2023-06-23T00:52:19.579 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-primax.ko 2023-06-23T00:52:19.579 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-rmi.ko 2023-06-23T00:52:19.579 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-roccat-arvo.ko 2023-06-23T00:52:19.580 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-roccat-common.ko 2023-06-23T00:52:19.580 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-roccat-isku.ko 2023-06-23T00:52:19.580 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-roccat-kone.ko 2023-06-23T00:52:19.580 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-06-23T00:52:19.580 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-roccat-konepure.ko 2023-06-23T00:52:19.580 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-06-23T00:52:19.580 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-roccat-lua.ko 2023-06-23T00:52:19.581 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-roccat-pyra.ko 2023-06-23T00:52:19.581 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-roccat-ryos.ko 2023-06-23T00:52:19.581 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-roccat-savu.ko 2023-06-23T00:52:19.581 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-roccat.ko 2023-06-23T00:52:19.581 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-saitek.ko 2023-06-23T00:52:19.581 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-samsung.ko 2023-06-23T00:52:19.581 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-sensor-hub.ko 2023-06-23T00:52:19.581 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-sjoy.ko 2023-06-23T00:52:19.582 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-sony.ko 2023-06-23T00:52:19.582 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-speedlink.ko 2023-06-23T00:52:19.582 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-steelseries.ko 2023-06-23T00:52:19.582 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-sunplus.ko 2023-06-23T00:52:19.582 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-thrustmaster.ko 2023-06-23T00:52:19.582 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-tivo.ko 2023-06-23T00:52:19.582 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-tmff.ko 2023-06-23T00:52:19.582 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-topseed.ko 2023-06-23T00:52:19.583 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-twinhan.ko 2023-06-23T00:52:19.583 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-uclogic.ko 2023-06-23T00:52:19.583 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-waltop.ko 2023-06-23T00:52:19.583 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-wiimote.ko 2023-06-23T00:52:19.583 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-xinmo.ko 2023-06-23T00:52:19.583 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-zpff.ko 2023-06-23T00:52:19.583 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-zydacron.ko 2023-06-23T00:52:19.583 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/uhid.ko 2023-06-23T00:52:19.584 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/wacom.ko 2023-06-23T00:52:19.584 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hv 2023-06-23T00:52:19.584 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hv/hv_balloon.ko 2023-06-23T00:52:19.584 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hv/hv_utils.ko 2023-06-23T00:52:19.584 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hv/hv_vmbus.ko 2023-06-23T00:52:19.584 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon 2023-06-23T00:52:19.584 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/abituguru.ko 2023-06-23T00:52:19.584 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/abituguru3.ko 2023-06-23T00:52:19.585 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/acpi_power_meter.ko 2023-06-23T00:52:19.585 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/ad7414.ko 2023-06-23T00:52:19.585 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/ad7418.ko 2023-06-23T00:52:19.585 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/adc128d818.ko 2023-06-23T00:52:19.585 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/adm1025.ko 2023-06-23T00:52:19.585 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/adm1026.ko 2023-06-23T00:52:19.585 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/adm1029.ko 2023-06-23T00:52:19.586 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/adm1031.ko 2023-06-23T00:52:19.586 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/adm9240.ko 2023-06-23T00:52:19.586 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/ads7828.ko 2023-06-23T00:52:19.586 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/adt7410.ko 2023-06-23T00:52:19.586 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/adt7411.ko 2023-06-23T00:52:19.586 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/adt7462.ko 2023-06-23T00:52:19.586 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/adt7470.ko 2023-06-23T00:52:19.586 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/adt7475.ko 2023-06-23T00:52:19.587 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/adt7x10.ko 2023-06-23T00:52:19.587 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/amc6821.ko 2023-06-23T00:52:19.587 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/applesmc.ko 2023-06-23T00:52:19.587 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/asb100.ko 2023-06-23T00:52:19.587 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/asc7621.ko 2023-06-23T00:52:19.587 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/asus_atk0110.ko 2023-06-23T00:52:19.587 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/atxp1.ko 2023-06-23T00:52:19.587 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/coretemp.ko 2023-06-23T00:52:19.588 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-06-23T00:52:19.588 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/dme1737.ko 2023-06-23T00:52:19.588 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/ds1621.ko 2023-06-23T00:52:19.588 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/ds620.ko 2023-06-23T00:52:19.588 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/emc1403.ko 2023-06-23T00:52:19.588 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/emc6w201.ko 2023-06-23T00:52:19.589 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/f71805f.ko 2023-06-23T00:52:19.589 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/f71882fg.ko 2023-06-23T00:52:19.589 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/f75375s.ko 2023-06-23T00:52:19.589 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/fam15h_power.ko 2023-06-23T00:52:19.589 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/fschmd.ko 2023-06-23T00:52:19.589 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/g760a.ko 2023-06-23T00:52:19.589 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/g762.ko 2023-06-23T00:52:19.589 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/gl518sm.ko 2023-06-23T00:52:19.590 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/gl520sm.ko 2023-06-23T00:52:19.590 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/hwmon-vid.ko 2023-06-23T00:52:19.590 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/i5500_temp.ko 2023-06-23T00:52:19.590 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/i5k_amb.ko 2023-06-23T00:52:19.590 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/ibmaem.ko 2023-06-23T00:52:19.590 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/ibmpex.ko 2023-06-23T00:52:19.590 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/ina209.ko 2023-06-23T00:52:19.590 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/ina2xx.ko 2023-06-23T00:52:19.591 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/it87.ko 2023-06-23T00:52:19.591 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/k10temp.ko 2023-06-23T00:52:19.591 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/k8temp.ko 2023-06-23T00:52:19.591 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/lineage-pem.ko 2023-06-23T00:52:19.591 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/lm63.ko 2023-06-23T00:52:19.591 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/lm73.ko 2023-06-23T00:52:19.591 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/lm75.ko 2023-06-23T00:52:19.591 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/lm77.ko 2023-06-23T00:52:19.592 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/lm78.ko 2023-06-23T00:52:19.592 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/lm80.ko 2023-06-23T00:52:19.592 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/lm83.ko 2023-06-23T00:52:19.592 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/lm85.ko 2023-06-23T00:52:19.592 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/lm87.ko 2023-06-23T00:52:19.592 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/lm90.ko 2023-06-23T00:52:19.592 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/lm92.ko 2023-06-23T00:52:19.592 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/lm93.ko 2023-06-23T00:52:19.593 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/lm95234.ko 2023-06-23T00:52:19.593 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/lm95241.ko 2023-06-23T00:52:19.593 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/lm95245.ko 2023-06-23T00:52:19.593 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/ltc2945.ko 2023-06-23T00:52:19.593 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/ltc4151.ko 2023-06-23T00:52:19.593 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/ltc4215.ko 2023-06-23T00:52:19.593 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/ltc4222.ko 2023-06-23T00:52:19.594 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/ltc4245.ko 2023-06-23T00:52:19.594 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/ltc4260.ko 2023-06-23T00:52:19.594 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/ltc4261.ko 2023-06-23T00:52:19.594 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/max16065.ko 2023-06-23T00:52:19.594 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/max1619.ko 2023-06-23T00:52:19.594 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/max1668.ko 2023-06-23T00:52:19.594 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/max197.ko 2023-06-23T00:52:19.594 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/max6639.ko 2023-06-23T00:52:19.595 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/max6650.ko 2023-06-23T00:52:19.595 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/max6697.ko 2023-06-23T00:52:19.595 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/mcp3021.ko 2023-06-23T00:52:19.595 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/nct6683.ko 2023-06-23T00:52:19.595 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/nct6775-core.ko 2023-06-23T00:52:19.595 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/nct6775.ko 2023-06-23T00:52:19.595 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/nct7802.ko 2023-06-23T00:52:19.595 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/pc87360.ko 2023-06-23T00:52:19.596 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/pc87427.ko 2023-06-23T00:52:19.596 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/pcf8591.ko 2023-06-23T00:52:19.596 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/pmbus 2023-06-23T00:52:19.596 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-06-23T00:52:19.596 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-06-23T00:52:19.596 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-06-23T00:52:19.596 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/pmbus/max16064.ko 2023-06-23T00:52:19.596 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/pmbus/max34440.ko 2023-06-23T00:52:19.597 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/pmbus/max8688.ko 2023-06-23T00:52:19.597 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-06-23T00:52:19.597 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-06-23T00:52:19.597 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-06-23T00:52:19.597 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-06-23T00:52:19.597 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-06-23T00:52:19.597 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-06-23T00:52:19.597 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/powr1220.ko 2023-06-23T00:52:19.598 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/sch5627.ko 2023-06-23T00:52:19.598 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/sch5636.ko 2023-06-23T00:52:19.598 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/sch56xx-common.ko 2023-06-23T00:52:19.598 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/sht15.ko 2023-06-23T00:52:19.598 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/sht21.ko 2023-06-23T00:52:19.598 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/shtc1.ko 2023-06-23T00:52:19.598 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/sis5595.ko 2023-06-23T00:52:19.598 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/smsc47b397.ko 2023-06-23T00:52:19.599 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/smsc47m1.ko 2023-06-23T00:52:19.599 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/smsc47m192.ko 2023-06-23T00:52:19.599 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/thmc50.ko 2023-06-23T00:52:19.599 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/tmp102.ko 2023-06-23T00:52:19.599 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/tmp103.ko 2023-06-23T00:52:19.599 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/tmp401.ko 2023-06-23T00:52:19.599 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/tmp421.ko 2023-06-23T00:52:19.599 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/via-cputemp.ko 2023-06-23T00:52:19.600 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/via686a.ko 2023-06-23T00:52:19.600 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/vt1211.ko 2023-06-23T00:52:19.600 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/vt8231.ko 2023-06-23T00:52:19.600 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/w83627ehf.ko 2023-06-23T00:52:19.600 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/w83627hf.ko 2023-06-23T00:52:19.600 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/w83781d.ko 2023-06-23T00:52:19.600 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/w83791d.ko 2023-06-23T00:52:19.601 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/w83792d.ko 2023-06-23T00:52:19.601 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/w83793.ko 2023-06-23T00:52:19.601 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/w83795.ko 2023-06-23T00:52:19.601 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/w83l785ts.ko 2023-06-23T00:52:19.601 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/w83l786ng.ko 2023-06-23T00:52:19.601 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c 2023-06-23T00:52:19.601 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/algos 2023-06-23T00:52:19.601 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-06-23T00:52:19.602 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-06-23T00:52:19.602 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses 2023-06-23T00:52:19.602 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-06-23T00:52:19.602 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-06-23T00:52:19.602 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-06-23T00:52:19.602 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-06-23T00:52:19.602 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-06-23T00:52:19.602 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-06-23T00:52:19.603 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-06-23T00:52:19.603 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-06-23T00:52:19.603 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-i801.ko 2023-06-23T00:52:19.603 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-isch.ko 2023-06-23T00:52:19.603 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-06-23T00:52:19.603 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-06-23T00:52:19.603 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-06-23T00:52:19.603 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-parport.ko 2023-06-23T00:52:19.604 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-06-23T00:52:19.604 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-06-23T00:52:19.604 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-06-23T00:52:19.604 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-06-23T00:52:19.604 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-06-23T00:52:19.604 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-06-23T00:52:19.604 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-via.ko 2023-06-23T00:52:19.605 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-06-23T00:52:19.605 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-06-23T00:52:19.605 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/i2c-dev.ko 2023-06-23T00:52:19.605 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/i2c-mux.ko 2023-06-23T00:52:19.605 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-06-23T00:52:19.605 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/i2c-smbus.ko 2023-06-23T00:52:19.605 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/i2c-stub.ko 2023-06-23T00:52:19.605 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband 2023-06-23T00:52:19.606 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/core 2023-06-23T00:52:19.606 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/core/ib_cm.ko 2023-06-23T00:52:19.606 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/core/ib_core.ko 2023-06-23T00:52:19.606 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/core/ib_umad.ko 2023-06-23T00:52:19.606 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-06-23T00:52:19.606 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/core/iw_cm.ko 2023-06-23T00:52:19.606 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/core/rdma_cm.ko 2023-06-23T00:52:19.606 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-06-23T00:52:19.607 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/hw 2023-06-23T00:52:19.607 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/hw/cxgb4 2023-06-23T00:52:19.607 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-06-23T00:52:19.607 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/hw/mlx4 2023-06-23T00:52:19.607 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-06-23T00:52:19.607 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/hw/mlx5 2023-06-23T00:52:19.607 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-06-23T00:52:19.607 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/hw/mthca 2023-06-23T00:52:19.608 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-06-23T00:52:19.608 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/ulp 2023-06-23T00:52:19.608 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/ulp/ipoib 2023-06-23T00:52:19.608 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-06-23T00:52:19.608 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/ulp/iser 2023-06-23T00:52:19.608 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-06-23T00:52:19.608 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/ulp/isert 2023-06-23T00:52:19.609 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-06-23T00:52:19.609 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/ulp/srp 2023-06-23T00:52:19.609 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-06-23T00:52:19.609 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/ulp/srpt 2023-06-23T00:52:19.609 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-06-23T00:52:19.609 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input 2023-06-23T00:52:19.609 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/gameport 2023-06-23T00:52:19.609 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-06-23T00:52:19.610 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/gameport/fm801-gp.ko 2023-06-23T00:52:19.610 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/gameport/gameport.ko 2023-06-23T00:52:19.610 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/gameport/lightning.ko 2023-06-23T00:52:19.610 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/gameport/ns558.ko 2023-06-23T00:52:19.610 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/joydev.ko 2023-06-23T00:52:19.610 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/keyboard 2023-06-23T00:52:19.610 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/keyboard/gpio_keys.ko 2023-06-23T00:52:19.610 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/mouse 2023-06-23T00:52:19.611 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/mouse/appletouch.ko 2023-06-23T00:52:19.611 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/mouse/bcm5974.ko 2023-06-23T00:52:19.611 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/mouse/cyapatp.ko 2023-06-23T00:52:19.611 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/mouse/elan_i2c.ko 2023-06-23T00:52:19.611 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/mouse/sermouse.ko 2023-06-23T00:52:19.611 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-06-23T00:52:19.611 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/mouse/synaptics_usb.ko 2023-06-23T00:52:19.611 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/mouse/vsxxxaa.ko 2023-06-23T00:52:19.612 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/rmi4 2023-06-23T00:52:19.612 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/rmi4/rmi_core.ko 2023-06-23T00:52:19.612 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/serio 2023-06-23T00:52:19.612 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/serio/altera_ps2.ko 2023-06-23T00:52:19.612 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/serio/arc_ps2.ko 2023-06-23T00:52:19.612 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-06-23T00:52:19.612 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/serio/serio_raw.ko 2023-06-23T00:52:19.613 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/sparse-keymap.ko 2023-06-23T00:52:19.613 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/iommu 2023-06-23T00:52:19.613 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/iommu/amd 2023-06-23T00:52:19.613 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/iommu/amd/iommu_v2.ko 2023-06-23T00:52:19.613 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/leds 2023-06-23T00:52:19.613 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/leds/leds-blinkm.ko 2023-06-23T00:52:19.613 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/leds/leds-lm3530.ko 2023-06-23T00:52:19.613 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/leds/leds-lp3944.ko 2023-06-23T00:52:19.614 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/leds/leds-ss4200.ko 2023-06-23T00:52:19.614 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/leds/trigger 2023-06-23T00:52:19.614 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-06-23T00:52:19.614 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-06-23T00:52:19.614 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-06-23T00:52:19.614 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-06-23T00:52:19.614 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-06-23T00:52:19.614 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-06-23T00:52:19.615 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-06-23T00:52:19.615 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-06-23T00:52:19.615 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md 2023-06-23T00:52:19.615 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/bcache 2023-06-23T00:52:19.615 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/bcache/bcache.ko 2023-06-23T00:52:19.615 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/dm-bio-prison.ko 2023-06-23T00:52:19.615 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/dm-cache-smq.ko 2023-06-23T00:52:19.616 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/dm-cache.ko 2023-06-23T00:52:19.616 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/dm-crypt.ko 2023-06-23T00:52:19.616 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/dm-delay.ko 2023-06-23T00:52:19.616 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/dm-flakey.ko 2023-06-23T00:52:19.616 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/dm-log-userspace.ko 2023-06-23T00:52:19.616 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/dm-multipath.ko 2023-06-23T00:52:19.616 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/dm-queue-length.ko 2023-06-23T00:52:19.616 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/dm-raid.ko 2023-06-23T00:52:19.617 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/dm-round-robin.ko 2023-06-23T00:52:19.617 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/dm-service-time.ko 2023-06-23T00:52:19.617 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/dm-switch.ko 2023-06-23T00:52:19.617 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/dm-thin-pool.ko 2023-06-23T00:52:19.617 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/dm-verity.ko 2023-06-23T00:52:19.617 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/faulty.ko 2023-06-23T00:52:19.617 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/linear.ko 2023-06-23T00:52:19.617 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/multipath.ko 2023-06-23T00:52:19.618 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/persistent-data 2023-06-23T00:52:19.618 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-06-23T00:52:19.618 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/raid0.ko 2023-06-23T00:52:19.618 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/raid1.ko 2023-06-23T00:52:19.618 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/raid10.ko 2023-06-23T00:52:19.618 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/raid456.ko 2023-06-23T00:52:19.618 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/message 2023-06-23T00:52:19.619 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/message/fusion 2023-06-23T00:52:19.619 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/message/fusion/mptbase.ko 2023-06-23T00:52:19.619 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/message/fusion/mptctl.ko 2023-06-23T00:52:19.619 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/message/fusion/mptfc.ko 2023-06-23T00:52:19.619 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/message/fusion/mptlan.ko 2023-06-23T00:52:19.619 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/message/fusion/mptsas.ko 2023-06-23T00:52:19.619 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/message/fusion/mptscsih.ko 2023-06-23T00:52:19.619 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/message/fusion/mptspi.ko 2023-06-23T00:52:19.620 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/mfd 2023-06-23T00:52:19.620 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/mfd/lpc_ich.ko 2023-06-23T00:52:19.620 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/mfd/lpc_sch.ko 2023-06-23T00:52:19.620 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/mfd/mfd-core.ko 2023-06-23T00:52:19.620 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/mfd/sm501.ko 2023-06-23T00:52:19.620 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/mfd/viperboard.ko 2023-06-23T00:52:19.620 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/mfd/vx855.ko 2023-06-23T00:52:19.620 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/mfd/wl1273-core.ko 2023-06-23T00:52:19.621 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc 2023-06-23T00:52:19.621 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/altera-stapl 2023-06-23T00:52:19.621 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-06-23T00:52:19.621 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/apds9802als.ko 2023-06-23T00:52:19.621 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/apds990x.ko 2023-06-23T00:52:19.621 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/bh1770glc.ko 2023-06-23T00:52:19.621 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/cb710 2023-06-23T00:52:19.621 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/cb710/cb710.ko 2023-06-23T00:52:19.622 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/eeprom 2023-06-23T00:52:19.622 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/eeprom/at24.ko 2023-06-23T00:52:19.622 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/eeprom/eeprom.ko 2023-06-23T00:52:19.622 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-06-23T00:52:19.622 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/eeprom/max6875.ko 2023-06-23T00:52:19.622 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/enclosure.ko 2023-06-23T00:52:19.622 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/hpilo.ko 2023-06-23T00:52:19.623 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/isl29003.ko 2023-06-23T00:52:19.623 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/isl29020.ko 2023-06-23T00:52:19.623 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/lis3lv02d 2023-06-23T00:52:19.623 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-06-23T00:52:19.623 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-06-23T00:52:19.623 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/mei 2023-06-23T00:52:19.623 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/mei/mei-me.ko 2023-06-23T00:52:19.623 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/mei/mei-txe.ko 2023-06-23T00:52:19.624 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/mei/mei.ko 2023-06-23T00:52:19.624 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/sgi-gru 2023-06-23T00:52:19.624 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/sgi-gru/gru.ko 2023-06-23T00:52:19.624 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/sgi-xp 2023-06-23T00:52:19.624 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/sgi-xp/xp.ko 2023-06-23T00:52:19.624 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/sgi-xp/xpc.ko 2023-06-23T00:52:19.624 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-06-23T00:52:19.624 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/tifm_7xx1.ko 2023-06-23T00:52:19.625 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/tifm_core.ko 2023-06-23T00:52:19.625 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/tsl2550.ko 2023-06-23T00:52:19.625 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/vmw_balloon.ko 2023-06-23T00:52:19.625 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/vmw_vmci 2023-06-23T00:52:19.625 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-06-23T00:52:19.625 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net 2023-06-23T00:52:19.625 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/bonding 2023-06-23T00:52:19.625 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/bonding/bonding.ko 2023-06-23T00:52:19.626 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/dsa 2023-06-23T00:52:19.626 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/dsa/b53 2023-06-23T00:52:19.626 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/dsa/b53/b53_common.ko 2023-06-23T00:52:19.626 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/dsa/bcm-sf2.ko 2023-06-23T00:52:19.626 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/dsa/mv88e6060.ko 2023-06-23T00:52:19.626 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/dsa/mv88e6xxx 2023-06-23T00:52:19.626 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-06-23T00:52:19.626 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/dummy.ko 2023-06-23T00:52:19.627 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/eql.ko 2023-06-23T00:52:19.627 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet 2023-06-23T00:52:19.627 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/3com 2023-06-23T00:52:19.627 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-06-23T00:52:19.627 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-06-23T00:52:19.627 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-06-23T00:52:19.627 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-06-23T00:52:19.628 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/8390 2023-06-23T00:52:19.628 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/8390/8390.ko 2023-06-23T00:52:19.628 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-06-23T00:52:19.628 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-06-23T00:52:19.628 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-06-23T00:52:19.628 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/adaptec 2023-06-23T00:52:19.628 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-06-23T00:52:19.628 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/agere 2023-06-23T00:52:19.629 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/agere/et131x.ko 2023-06-23T00:52:19.629 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/alteon 2023-06-23T00:52:19.629 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-06-23T00:52:19.629 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/altera 2023-06-23T00:52:19.629 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-06-23T00:52:19.629 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/amd 2023-06-23T00:52:19.629 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-06-23T00:52:19.629 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-06-23T00:52:19.630 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-06-23T00:52:19.630 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/atheros 2023-06-23T00:52:19.630 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/atheros/alx 2023-06-23T00:52:19.630 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-06-23T00:52:19.630 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/atheros/atl1c 2023-06-23T00:52:19.630 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-06-23T00:52:19.630 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/atheros/atl1e 2023-06-23T00:52:19.630 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-06-23T00:52:19.631 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/atheros/atlx 2023-06-23T00:52:19.631 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-06-23T00:52:19.631 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-06-23T00:52:19.631 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/broadcom 2023-06-23T00:52:19.631 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-06-23T00:52:19.631 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-06-23T00:52:19.631 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-06-23T00:52:19.632 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-06-23T00:52:19.632 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-06-23T00:52:19.632 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/broadcom/genet 2023-06-23T00:52:19.632 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-06-23T00:52:19.632 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-06-23T00:52:19.632 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/brocade 2023-06-23T00:52:19.632 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/brocade/bna 2023-06-23T00:52:19.632 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-06-23T00:52:19.633 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/chelsio 2023-06-23T00:52:19.633 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/chelsio/cxgb 2023-06-23T00:52:19.633 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-06-23T00:52:19.633 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-06-23T00:52:19.633 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-06-23T00:52:19.633 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-06-23T00:52:19.633 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-06-23T00:52:19.633 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-06-23T00:52:19.634 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-06-23T00:52:19.634 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-06-23T00:52:19.634 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-06-23T00:52:19.634 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/cisco 2023-06-23T00:52:19.634 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/cisco/enic 2023-06-23T00:52:19.634 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-06-23T00:52:19.634 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/dec 2023-06-23T00:52:19.634 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/dec/tulip 2023-06-23T00:52:19.635 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-06-23T00:52:19.635 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-06-23T00:52:19.635 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-06-23T00:52:19.635 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-06-23T00:52:19.635 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-06-23T00:52:19.635 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-06-23T00:52:19.635 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/dlink 2023-06-23T00:52:19.636 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-06-23T00:52:19.636 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-06-23T00:52:19.636 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/dnet.ko 2023-06-23T00:52:19.636 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/emulex 2023-06-23T00:52:19.636 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/emulex/benet 2023-06-23T00:52:19.636 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-06-23T00:52:19.636 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/ethoc.ko 2023-06-23T00:52:19.636 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/fealnx.ko 2023-06-23T00:52:19.637 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel 2023-06-23T00:52:19.637 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel/e100.ko 2023-06-23T00:52:19.637 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel/e1000 2023-06-23T00:52:19.637 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-06-23T00:52:19.637 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel/e1000e 2023-06-23T00:52:19.637 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-06-23T00:52:19.637 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel/fm10k 2023-06-23T00:52:19.637 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-06-23T00:52:19.638 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel/i40e 2023-06-23T00:52:19.638 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-06-23T00:52:19.638 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel/iavf 2023-06-23T00:52:19.638 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-06-23T00:52:19.640 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel/igb 2023-06-23T00:52:19.640 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-06-23T00:52:19.640 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel/igbvf 2023-06-23T00:52:19.641 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-06-23T00:52:19.641 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel/ixgbe 2023-06-23T00:52:19.641 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-06-23T00:52:19.641 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel/ixgbevf 2023-06-23T00:52:19.641 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-06-23T00:52:19.641 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/jme.ko 2023-06-23T00:52:19.641 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/marvell 2023-06-23T00:52:19.641 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-06-23T00:52:19.642 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/marvell/skge.ko 2023-06-23T00:52:19.642 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-06-23T00:52:19.642 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/mellanox 2023-06-23T00:52:19.642 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/mellanox/mlx4 2023-06-23T00:52:19.642 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-06-23T00:52:19.642 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-06-23T00:52:19.642 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/mellanox/mlx5 2023-06-23T00:52:19.642 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-06-23T00:52:19.643 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-06-23T00:52:19.643 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/micrel 2023-06-23T00:52:19.643 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-06-23T00:52:19.643 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/myricom 2023-06-23T00:52:19.643 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/myricom/myri10ge 2023-06-23T00:52:19.643 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-06-23T00:52:19.643 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/natsemi 2023-06-23T00:52:19.644 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-06-23T00:52:19.644 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-06-23T00:52:19.644 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/neterion 2023-06-23T00:52:19.644 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-06-23T00:52:19.644 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/nvidia 2023-06-23T00:52:19.644 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-06-23T00:52:19.644 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/packetengines 2023-06-23T00:52:19.644 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-06-23T00:52:19.645 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-06-23T00:52:19.645 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/qlogic 2023-06-23T00:52:19.645 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/qlogic/netxen 2023-06-23T00:52:19.645 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-06-23T00:52:19.645 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-06-23T00:52:19.645 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-06-23T00:52:19.645 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-06-23T00:52:19.645 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/rdc 2023-06-23T00:52:19.646 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-06-23T00:52:19.646 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/realtek 2023-06-23T00:52:19.646 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-06-23T00:52:19.646 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-06-23T00:52:19.646 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/realtek/atp.ko 2023-06-23T00:52:19.646 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-06-23T00:52:19.646 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/rocker 2023-06-23T00:52:19.647 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-06-23T00:52:19.647 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/sfc 2023-06-23T00:52:19.647 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-06-23T00:52:19.647 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/silan 2023-06-23T00:52:19.647 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-06-23T00:52:19.647 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/sis 2023-06-23T00:52:19.647 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/sis/sis190.ko 2023-06-23T00:52:19.647 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/sis/sis900.ko 2023-06-23T00:52:19.648 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/smsc 2023-06-23T00:52:19.648 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-06-23T00:52:19.648 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-06-23T00:52:19.648 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-06-23T00:52:19.648 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-06-23T00:52:19.648 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/stmicro 2023-06-23T00:52:19.648 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/stmicro/stmmac 2023-06-23T00:52:19.648 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-06-23T00:52:19.649 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-06-23T00:52:19.649 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/sun 2023-06-23T00:52:19.649 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/sun/cassini.ko 2023-06-23T00:52:19.649 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/sun/niu.ko 2023-06-23T00:52:19.649 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/sun/sungem.ko 2023-06-23T00:52:19.649 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-06-23T00:52:19.649 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/tehuti 2023-06-23T00:52:19.650 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-06-23T00:52:19.650 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/ti 2023-06-23T00:52:19.650 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/ti/tlan.ko 2023-06-23T00:52:19.650 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/via 2023-06-23T00:52:19.650 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-06-23T00:52:19.650 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-06-23T00:52:19.650 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/wiznet 2023-06-23T00:52:19.650 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-06-23T00:52:19.651 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-06-23T00:52:19.651 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/xircom 2023-06-23T00:52:19.651 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-06-23T00:52:19.651 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/geneve.ko 2023-06-23T00:52:19.651 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/hyperv 2023-06-23T00:52:19.651 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-06-23T00:52:19.651 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ieee802154 2023-06-23T00:52:19.651 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ieee802154/fakelb.ko 2023-06-23T00:52:19.652 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ifb.ko 2023-06-23T00:52:19.652 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ipvlan 2023-06-23T00:52:19.652 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ipvlan/ipvlan.ko 2023-06-23T00:52:19.652 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/macvlan.ko 2023-06-23T00:52:19.652 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/macvtap.ko 2023-06-23T00:52:19.652 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/mdio 2023-06-23T00:52:19.652 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/mdio.ko 2023-06-23T00:52:19.652 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-06-23T00:52:19.653 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-06-23T00:52:19.653 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/mii.ko 2023-06-23T00:52:19.653 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/net_failover.ko 2023-06-23T00:52:19.653 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/netconsole.ko 2023-06-23T00:52:19.653 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/nlmon.ko 2023-06-23T00:52:19.653 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/pcs 2023-06-23T00:52:19.653 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/pcs/pcs-altera-tse.ko 2023-06-23T00:52:19.654 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-06-23T00:52:19.654 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy 2023-06-23T00:52:19.654 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/amd.ko 2023-06-23T00:52:19.654 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-06-23T00:52:19.654 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-06-23T00:52:19.654 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/bcm7xxx.ko 2023-06-23T00:52:19.654 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/bcm87xx.ko 2023-06-23T00:52:19.654 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/broadcom.ko 2023-06-23T00:52:19.655 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/cicada.ko 2023-06-23T00:52:19.655 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/davicom.ko 2023-06-23T00:52:19.655 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/dp83640.ko 2023-06-23T00:52:19.655 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/et1011c.ko 2023-06-23T00:52:19.655 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/icplus.ko 2023-06-23T00:52:19.655 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/lxt.ko 2023-06-23T00:52:19.655 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/marvell.ko 2023-06-23T00:52:19.655 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/micrel.ko 2023-06-23T00:52:19.656 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/national.ko 2023-06-23T00:52:19.656 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/phylink.ko 2023-06-23T00:52:19.656 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/qsemi.ko 2023-06-23T00:52:19.656 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/realtek.ko 2023-06-23T00:52:19.656 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/smsc.ko 2023-06-23T00:52:19.656 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/ste10Xp.ko 2023-06-23T00:52:19.656 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/vitesse.ko 2023-06-23T00:52:19.657 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ppp 2023-06-23T00:52:19.657 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ppp/bsd_comp.ko 2023-06-23T00:52:19.657 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ppp/ppp_async.ko 2023-06-23T00:52:19.657 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ppp/ppp_deflate.ko 2023-06-23T00:52:19.657 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ppp/ppp_generic.ko 2023-06-23T00:52:19.657 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ppp/ppp_mppe.ko 2023-06-23T00:52:19.658 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ppp/ppp_synctty.ko 2023-06-23T00:52:19.658 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ppp/pppoe.ko 2023-06-23T00:52:19.658 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ppp/pppox.ko 2023-06-23T00:52:19.658 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ppp/pptp.ko 2023-06-23T00:52:19.658 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/slip 2023-06-23T00:52:19.658 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/slip/slhc.ko 2023-06-23T00:52:19.658 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/slip/slip.ko 2023-06-23T00:52:19.658 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/sungem_phy.ko 2023-06-23T00:52:19.659 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/tap.ko 2023-06-23T00:52:19.659 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/team 2023-06-23T00:52:19.659 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/team/team.ko 2023-06-23T00:52:19.659 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/team/team_mode_activebackup.ko 2023-06-23T00:52:19.659 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/team/team_mode_broadcast.ko 2023-06-23T00:52:19.660 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-06-23T00:52:19.660 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/team/team_mode_random.ko 2023-06-23T00:52:19.660 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-06-23T00:52:19.660 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/tun.ko 2023-06-23T00:52:19.660 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/veth.ko 2023-06-23T00:52:19.660 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/virtio_net.ko 2023-06-23T00:52:19.660 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/vmxnet3 2023-06-23T00:52:19.660 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-06-23T00:52:19.661 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/vxlan 2023-06-23T00:52:19.661 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/vxlan/vxlan.ko 2023-06-23T00:52:19.661 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/xen-netback 2023-06-23T00:52:19.661 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/xen-netback/xen-netback.ko 2023-06-23T00:52:19.661 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/xen-netfront.ko 2023-06-23T00:52:19.661 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ntb 2023-06-23T00:52:19.661 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ntb/ntb.ko 2023-06-23T00:52:19.661 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/nvme 2023-06-23T00:52:19.662 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/nvme/host 2023-06-23T00:52:19.662 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/nvme/host/nvme-core.ko 2023-06-23T00:52:19.662 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/nvme/host/nvme.ko 2023-06-23T00:52:19.662 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/parport 2023-06-23T00:52:19.662 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/parport/parport.ko 2023-06-23T00:52:19.662 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/parport/parport_cs.ko 2023-06-23T00:52:19.662 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/parport/parport_pc.ko 2023-06-23T00:52:19.662 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/parport/parport_serial.ko 2023-06-23T00:52:19.663 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/pci 2023-06-23T00:52:19.663 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/pci/hotplug 2023-06-23T00:52:19.663 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-06-23T00:52:19.663 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/pci/pcie 2023-06-23T00:52:19.663 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/pci/pcie/aer_inject.ko 2023-06-23T00:52:19.663 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/pci/xen-pcifront.ko 2023-06-23T00:52:19.663 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/pcmcia 2023-06-23T00:52:19.664 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/pcmcia/i82092.ko 2023-06-23T00:52:19.664 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/pcmcia/pd6729.ko 2023-06-23T00:52:19.664 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/pcmcia/yenta_socket.ko 2023-06-23T00:52:19.664 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/pinctrl 2023-06-23T00:52:19.664 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/pinctrl/intel 2023-06-23T00:52:19.664 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-06-23T00:52:19.664 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform 2023-06-23T00:52:19.664 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86 2023-06-23T00:52:19.665 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/acer-wmi.ko 2023-06-23T00:52:19.665 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/acerhdf.ko 2023-06-23T00:52:19.665 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/apple-gmux.ko 2023-06-23T00:52:19.665 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/asus-laptop.ko 2023-06-23T00:52:19.665 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-06-23T00:52:19.665 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/asus-wmi.ko 2023-06-23T00:52:19.665 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/classmate-laptop.ko 2023-06-23T00:52:19.665 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-06-23T00:52:19.666 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-06-23T00:52:19.666 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-06-23T00:52:19.666 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-06-23T00:52:19.666 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/hdaps.ko 2023-06-23T00:52:19.666 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/intel 2023-06-23T00:52:19.666 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-06-23T00:52:19.666 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/intel_ips.ko 2023-06-23T00:52:19.666 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/msi-wmi.ko 2023-06-23T00:52:19.667 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/mxm-wmi.ko 2023-06-23T00:52:19.667 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-06-23T00:52:19.667 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/samsung-laptop.ko 2023-06-23T00:52:19.667 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/samsung-q10.ko 2023-06-23T00:52:19.667 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/topstar-laptop.ko 2023-06-23T00:52:19.667 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-06-23T00:52:19.667 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/toshiba_haps.ko 2023-06-23T00:52:19.667 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/wmi-bmof.ko 2023-06-23T00:52:19.668 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/wmi.ko 2023-06-23T00:52:19.668 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/powercap 2023-06-23T00:52:19.668 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/powercap/intel_rapl_common.ko 2023-06-23T00:52:19.668 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/powercap/intel_rapl_msr.ko 2023-06-23T00:52:19.668 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/pps 2023-06-23T00:52:19.668 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/pps/clients 2023-06-23T00:52:19.668 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/pps/clients/pps-gpio.ko 2023-06-23T00:52:19.669 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/pps/clients/pps-ldisc.ko 2023-06-23T00:52:19.669 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/pps/clients/pps_parport.ko 2023-06-23T00:52:19.669 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/pps/pps_core.ko 2023-06-23T00:52:19.669 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ptp 2023-06-23T00:52:19.669 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ptp/ptp.ko 2023-06-23T00:52:19.669 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ptp/ptp_kvm.ko 2023-06-23T00:52:19.669 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc 2023-06-23T00:52:19.669 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-bq32k.ko 2023-06-23T00:52:19.670 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-bq4802.ko 2023-06-23T00:52:19.670 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-ds1286.ko 2023-06-23T00:52:19.670 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-ds1307.ko 2023-06-23T00:52:19.670 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-ds1374.ko 2023-06-23T00:52:19.670 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-ds1511.ko 2023-06-23T00:52:19.670 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-ds1553.ko 2023-06-23T00:52:19.670 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-ds1672.ko 2023-06-23T00:52:19.670 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-ds1742.ko 2023-06-23T00:52:19.671 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-ds2404.ko 2023-06-23T00:52:19.671 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-ds3232.ko 2023-06-23T00:52:19.671 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-em3027.ko 2023-06-23T00:52:19.671 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-fm3130.ko 2023-06-23T00:52:19.671 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-isl12022.ko 2023-06-23T00:52:19.671 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-isl1208.ko 2023-06-23T00:52:19.671 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-m41t80.ko 2023-06-23T00:52:19.671 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-m48t35.ko 2023-06-23T00:52:19.672 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-m48t59.ko 2023-06-23T00:52:19.672 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-max6900.ko 2023-06-23T00:52:19.672 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-msm6242.ko 2023-06-23T00:52:19.672 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-pcf2127.ko 2023-06-23T00:52:19.672 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-pcf85063.ko 2023-06-23T00:52:19.672 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-pcf8523.ko 2023-06-23T00:52:19.672 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-pcf8563.ko 2023-06-23T00:52:19.672 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-pcf8583.ko 2023-06-23T00:52:19.673 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-rp5c01.ko 2023-06-23T00:52:19.673 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-rs5c372.ko 2023-06-23T00:52:19.673 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-06-23T00:52:19.673 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-rx8025.ko 2023-06-23T00:52:19.673 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-rx8581.ko 2023-06-23T00:52:19.673 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-06-23T00:52:19.673 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-x1205.ko 2023-06-23T00:52:19.673 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi 2023-06-23T00:52:19.674 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/3w-9xxx.ko 2023-06-23T00:52:19.674 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/3w-sas.ko 2023-06-23T00:52:19.674 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/3w-xxxx.ko 2023-06-23T00:52:19.674 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/BusLogic.ko 2023-06-23T00:52:19.674 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/a100u2w.ko 2023-06-23T00:52:19.674 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/aacraid 2023-06-23T00:52:19.674 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/aacraid/aacraid.ko 2023-06-23T00:52:19.675 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/advansys.ko 2023-06-23T00:52:19.675 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/aic7xxx 2023-06-23T00:52:19.675 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-06-23T00:52:19.675 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-06-23T00:52:19.675 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/am53c974.ko 2023-06-23T00:52:19.675 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/arcmsr 2023-06-23T00:52:19.675 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-06-23T00:52:19.675 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/atp870u.ko 2023-06-23T00:52:19.676 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/be2iscsi 2023-06-23T00:52:19.676 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-06-23T00:52:19.676 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/bfa 2023-06-23T00:52:19.676 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/bfa/bfa.ko 2023-06-23T00:52:19.676 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/bnx2fc 2023-06-23T00:52:19.676 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-06-23T00:52:19.676 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/bnx2i 2023-06-23T00:52:19.676 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-06-23T00:52:19.677 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/ch.ko 2023-06-23T00:52:19.677 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/csiostor 2023-06-23T00:52:19.677 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/csiostor/csiostor.ko 2023-06-23T00:52:19.677 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/cxgbi 2023-06-23T00:52:19.677 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/cxgbi/cxgb3i 2023-06-23T00:52:19.677 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-06-23T00:52:19.677 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/cxgbi/cxgb4i 2023-06-23T00:52:19.677 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-06-23T00:52:19.678 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-06-23T00:52:19.678 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/dc395x.ko 2023-06-23T00:52:19.678 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/device_handler 2023-06-23T00:52:19.678 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-06-23T00:52:19.678 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-06-23T00:52:19.678 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-06-23T00:52:19.678 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-06-23T00:52:19.678 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/dmx3191d.ko 2023-06-23T00:52:19.679 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/esas2r 2023-06-23T00:52:19.679 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/esas2r/esas2r.ko 2023-06-23T00:52:19.679 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/esp_scsi.ko 2023-06-23T00:52:19.679 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/fcoe 2023-06-23T00:52:19.679 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/fcoe/fcoe.ko 2023-06-23T00:52:19.679 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-06-23T00:52:19.679 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/fnic 2023-06-23T00:52:19.679 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/fnic/fnic.ko 2023-06-23T00:52:19.680 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/hpsa.ko 2023-06-23T00:52:19.680 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/hptiop.ko 2023-06-23T00:52:19.680 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/hv_storvsc.ko 2023-06-23T00:52:19.680 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/initio.ko 2023-06-23T00:52:19.680 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/ipr.ko 2023-06-23T00:52:19.680 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/ips.ko 2023-06-23T00:52:19.680 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/isci 2023-06-23T00:52:19.680 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/isci/isci.ko 2023-06-23T00:52:19.681 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-06-23T00:52:19.681 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/iscsi_tcp.ko 2023-06-23T00:52:19.681 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/libfc 2023-06-23T00:52:19.681 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/libfc/libfc.ko 2023-06-23T00:52:19.681 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/libiscsi.ko 2023-06-23T00:52:19.681 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/libiscsi_tcp.ko 2023-06-23T00:52:19.681 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/libsas 2023-06-23T00:52:19.681 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/libsas/libsas.ko 2023-06-23T00:52:19.682 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/lpfc 2023-06-23T00:52:19.682 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/lpfc/lpfc.ko 2023-06-23T00:52:19.682 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/megaraid 2023-06-23T00:52:19.682 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/megaraid.ko 2023-06-23T00:52:19.682 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-06-23T00:52:19.682 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-06-23T00:52:19.682 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-06-23T00:52:19.683 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/mpt3sas 2023-06-23T00:52:19.683 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-06-23T00:52:19.683 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/mvsas 2023-06-23T00:52:19.683 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/mvsas/mvsas.ko 2023-06-23T00:52:19.683 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/mvumi.ko 2023-06-23T00:52:19.683 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/pm8001 2023-06-23T00:52:19.683 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-06-23T00:52:19.683 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/pmcraid.ko 2023-06-23T00:52:19.684 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/qla1280.ko 2023-06-23T00:52:19.684 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/qla2xxx 2023-06-23T00:52:19.684 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-06-23T00:52:19.684 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-06-23T00:52:19.684 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/qla4xxx 2023-06-23T00:52:19.684 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-06-23T00:52:19.684 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/raid_class.ko 2023-06-23T00:52:19.684 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/scsi_debug.ko 2023-06-23T00:52:19.685 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/scsi_transport_fc.ko 2023-06-23T00:52:19.685 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-06-23T00:52:19.685 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/scsi_transport_sas.ko 2023-06-23T00:52:19.685 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/scsi_transport_spi.ko 2023-06-23T00:52:19.685 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/scsi_transport_srp.ko 2023-06-23T00:52:19.685 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/ses.ko 2023-06-23T00:52:19.685 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/st.ko 2023-06-23T00:52:19.685 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/stex.ko 2023-06-23T00:52:19.686 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/sym53c8xx_2 2023-06-23T00:52:19.686 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-06-23T00:52:19.686 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/virtio_scsi.ko 2023-06-23T00:52:19.686 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/vmw_pvscsi.ko 2023-06-23T00:52:19.686 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/wd719x.ko 2023-06-23T00:52:19.686 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ssb 2023-06-23T00:52:19.686 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ssb/ssb.ko 2023-06-23T00:52:19.686 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/target 2023-06-23T00:52:19.687 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/target/iscsi 2023-06-23T00:52:19.687 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-06-23T00:52:19.687 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/target/loopback 2023-06-23T00:52:19.687 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/target/loopback/tcm_loop.ko 2023-06-23T00:52:19.687 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/target/target_core_file.ko 2023-06-23T00:52:19.687 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/target/target_core_iblock.ko 2023-06-23T00:52:19.687 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/target/target_core_mod.ko 2023-06-23T00:52:19.687 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/target/target_core_pscsi.ko 2023-06-23T00:52:19.688 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/target/target_core_user.ko 2023-06-23T00:52:19.688 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/target/tcm_fc 2023-06-23T00:52:19.688 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-06-23T00:52:19.688 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/thermal 2023-06-23T00:52:19.688 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/thermal/intel 2023-06-23T00:52:19.688 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/thermal/intel/int340x_thermal 2023-06-23T00:52:19.688 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-06-23T00:52:19.689 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-06-23T00:52:19.689 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-06-23T00:52:19.689 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-06-23T00:52:19.689 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-06-23T00:52:19.689 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-06-23T00:52:19.689 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-06-23T00:52:19.689 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-06-23T00:52:19.689 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-06-23T00:52:19.690 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-06-23T00:52:19.690 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-06-23T00:52:19.690 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-06-23T00:52:19.690 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-06-23T00:52:19.690 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-06-23T00:52:19.690 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-06-23T00:52:19.690 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/tty 2023-06-23T00:52:19.690 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/tty/ipwireless 2023-06-23T00:52:19.691 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-06-23T00:52:19.691 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/tty/n_gsm.ko 2023-06-23T00:52:19.691 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/tty/n_hdlc.ko 2023-06-23T00:52:19.691 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/tty/nozomi.ko 2023-06-23T00:52:19.691 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/tty/serial 2023-06-23T00:52:19.691 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/tty/serial/8250 2023-06-23T00:52:19.691 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-06-23T00:52:19.691 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/tty/serial/arc_uart.ko 2023-06-23T00:52:19.692 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/tty/serial/jsm 2023-06-23T00:52:19.692 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/tty/serial/jsm/jsm.ko 2023-06-23T00:52:19.692 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/tty/synclink_gt.ko 2023-06-23T00:52:19.692 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ufs 2023-06-23T00:52:19.692 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ufs/core 2023-06-23T00:52:19.692 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ufs/core/ufshcd-core.ko 2023-06-23T00:52:19.692 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ufs/host 2023-06-23T00:52:19.693 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-06-23T00:52:19.693 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/uio 2023-06-23T00:52:19.693 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/uio/uio.ko 2023-06-23T00:52:19.693 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/uio/uio_aec.ko 2023-06-23T00:52:19.693 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/uio/uio_cif.ko 2023-06-23T00:52:19.693 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/uio/uio_pci_generic.ko 2023-06-23T00:52:19.693 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/uio/uio_sercos3.ko 2023-06-23T00:52:19.693 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb 2023-06-23T00:52:19.694 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/class 2023-06-23T00:52:19.694 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/class/cdc-acm.ko 2023-06-23T00:52:19.694 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/class/cdc-wdm.ko 2023-06-23T00:52:19.694 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/class/usblp.ko 2023-06-23T00:52:19.694 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/class/usbtmc.ko 2023-06-23T00:52:19.694 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/host 2023-06-23T00:52:19.694 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/host/sl811-hcd.ko 2023-06-23T00:52:19.694 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/image 2023-06-23T00:52:19.695 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/image/mdc800.ko 2023-06-23T00:52:19.695 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/image/microtek.ko 2023-06-23T00:52:19.695 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/misc 2023-06-23T00:52:19.695 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/misc/adutux.ko 2023-06-23T00:52:19.695 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/misc/appledisplay.ko 2023-06-23T00:52:19.695 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/misc/emi26.ko 2023-06-23T00:52:19.695 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/misc/emi62.ko 2023-06-23T00:52:19.695 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/misc/ezusb.ko 2023-06-23T00:52:19.696 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/misc/idmouse.ko 2023-06-23T00:52:19.696 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/misc/iowarrior.ko 2023-06-23T00:52:19.696 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/misc/isight_firmware.ko 2023-06-23T00:52:19.696 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/misc/ldusb.ko 2023-06-23T00:52:19.696 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/misc/legousbtower.ko 2023-06-23T00:52:19.696 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/misc/sisusbvga 2023-06-23T00:52:19.696 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-06-23T00:52:19.697 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/misc/trancevibrator.ko 2023-06-23T00:52:19.697 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/misc/usb3503.ko 2023-06-23T00:52:19.697 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/misc/usblcd.ko 2023-06-23T00:52:19.697 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/misc/usbsevseg.ko 2023-06-23T00:52:19.697 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/misc/uss720.ko 2023-06-23T00:52:19.697 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/misc/yurex.ko 2023-06-23T00:52:19.697 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/phy 2023-06-23T00:52:19.697 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/phy/phy-generic.ko 2023-06-23T00:52:19.698 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial 2023-06-23T00:52:19.698 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/aircable.ko 2023-06-23T00:52:19.698 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/ark3116.ko 2023-06-23T00:52:19.698 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/belkin_sa.ko 2023-06-23T00:52:19.698 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/ch341.ko 2023-06-23T00:52:19.698 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/cp210x.ko 2023-06-23T00:52:19.698 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/cyberjack.ko 2023-06-23T00:52:19.698 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/cypress_m8.ko 2023-06-23T00:52:19.699 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/digi_acceleport.ko 2023-06-23T00:52:19.699 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/empeg.ko 2023-06-23T00:52:19.699 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/ftdi_sio.ko 2023-06-23T00:52:19.699 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/garmin_gps.ko 2023-06-23T00:52:19.699 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/io_edgeport.ko 2023-06-23T00:52:19.699 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/io_ti.ko 2023-06-23T00:52:19.699 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/ipaq.ko 2023-06-23T00:52:19.699 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/ipw.ko 2023-06-23T00:52:19.700 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/ir-usb.ko 2023-06-23T00:52:19.700 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-06-23T00:52:19.700 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/keyspan.ko 2023-06-23T00:52:19.700 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/keyspan_pda.ko 2023-06-23T00:52:19.700 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/kl5kusb105.ko 2023-06-23T00:52:19.700 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/kobil_sct.ko 2023-06-23T00:52:19.700 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/mct_u232.ko 2023-06-23T00:52:19.700 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/mos7720.ko 2023-06-23T00:52:19.701 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/mos7840.ko 2023-06-23T00:52:19.701 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/navman.ko 2023-06-23T00:52:19.701 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/omninet.ko 2023-06-23T00:52:19.701 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/opticon.ko 2023-06-23T00:52:19.701 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/option.ko 2023-06-23T00:52:19.701 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/oti6858.ko 2023-06-23T00:52:19.701 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/pl2303.ko 2023-06-23T00:52:19.701 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/qcaux.ko 2023-06-23T00:52:19.702 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/qcserial.ko 2023-06-23T00:52:19.702 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/quatech2.ko 2023-06-23T00:52:19.702 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/safe_serial.ko 2023-06-23T00:52:19.702 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/sierra.ko 2023-06-23T00:52:19.702 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/spcp8x5.ko 2023-06-23T00:52:19.702 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/ssu100.ko 2023-06-23T00:52:19.702 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/symbolserial.ko 2023-06-23T00:52:19.702 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-06-23T00:52:19.703 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-06-23T00:52:19.703 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/usb_debug.ko 2023-06-23T00:52:19.703 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/usb_wwan.ko 2023-06-23T00:52:19.703 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/visor.ko 2023-06-23T00:52:19.703 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/whiteheat.ko 2023-06-23T00:52:19.703 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/xsens_mt.ko 2023-06-23T00:52:19.703 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/storage 2023-06-23T00:52:19.703 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/storage/uas.ko 2023-06-23T00:52:19.704 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/storage/ums-alauda.ko 2023-06-23T00:52:19.704 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/storage/ums-cypress.ko 2023-06-23T00:52:19.704 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/storage/ums-datafab.ko 2023-06-23T00:52:19.704 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-06-23T00:52:19.704 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/storage/ums-freecom.ko 2023-06-23T00:52:19.704 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/storage/ums-isd200.ko 2023-06-23T00:52:19.704 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-06-23T00:52:19.704 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/storage/ums-karma.ko 2023-06-23T00:52:19.705 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/storage/ums-onetouch.ko 2023-06-23T00:52:19.705 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/storage/ums-realtek.ko 2023-06-23T00:52:19.705 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/storage/ums-sddr09.ko 2023-06-23T00:52:19.705 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/storage/ums-sddr55.ko 2023-06-23T00:52:19.705 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/storage/ums-usbat.ko 2023-06-23T00:52:19.705 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/storage/usb-storage.ko 2023-06-23T00:52:19.705 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/usbip 2023-06-23T00:52:19.705 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/usbip/usbip-core.ko 2023-06-23T00:52:19.706 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/usbip/usbip-host.ko 2023-06-23T00:52:19.706 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-06-23T00:52:19.706 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/vfio 2023-06-23T00:52:19.706 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/vfio/pci 2023-06-23T00:52:19.706 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-06-23T00:52:19.706 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/vfio/pci/vfio-pci.ko 2023-06-23T00:52:19.706 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/vfio/vfio.ko 2023-06-23T00:52:19.707 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-06-23T00:52:19.707 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/vhost 2023-06-23T00:52:19.707 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/vhost/vhost.ko 2023-06-23T00:52:19.707 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/vhost/vhost_iotlb.ko 2023-06-23T00:52:19.707 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/vhost/vhost_net.ko 2023-06-23T00:52:19.707 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/vhost/vhost_scsi.ko 2023-06-23T00:52:19.707 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/video 2023-06-23T00:52:19.707 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/video/backlight 2023-06-23T00:52:19.708 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/video/backlight/apple_bl.ko 2023-06-23T00:52:19.708 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/video/backlight/lcd.ko 2023-06-23T00:52:19.708 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/video/backlight/platform_lcd.ko 2023-06-23T00:52:19.708 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/virtio 2023-06-23T00:52:19.708 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/virtio/virtio.ko 2023-06-23T00:52:19.708 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/virtio/virtio_balloon.ko 2023-06-23T00:52:19.708 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/virtio/virtio_mmio.ko 2023-06-23T00:52:19.708 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/virtio/virtio_pci.ko 2023-06-23T00:52:19.709 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-06-23T00:52:19.709 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-06-23T00:52:19.709 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/virtio/virtio_ring.ko 2023-06-23T00:52:19.709 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/w1 2023-06-23T00:52:19.709 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/w1/masters 2023-06-23T00:52:19.709 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/w1/masters/ds2482.ko 2023-06-23T00:52:19.709 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/w1/masters/ds2490.ko 2023-06-23T00:52:19.709 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/w1/slaves 2023-06-23T00:52:19.710 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-06-23T00:52:19.710 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-06-23T00:52:19.710 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-06-23T00:52:19.710 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-06-23T00:52:19.710 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-06-23T00:52:19.710 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-06-23T00:52:19.710 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-06-23T00:52:19.710 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-06-23T00:52:19.711 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-06-23T00:52:19.711 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/w1/slaves/w1_smem.ko 2023-06-23T00:52:19.711 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/w1/slaves/w1_therm.ko 2023-06-23T00:52:19.711 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/w1/wire.ko 2023-06-23T00:52:19.711 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog 2023-06-23T00:52:19.711 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/alim1535_wdt.ko 2023-06-23T00:52:19.711 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/alim7101_wdt.ko 2023-06-23T00:52:19.711 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/f71808e_wdt.ko 2023-06-23T00:52:19.712 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/hpwdt.ko 2023-06-23T00:52:19.712 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/i6300esb.ko 2023-06-23T00:52:19.712 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-06-23T00:52:19.712 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/iTCO_wdt.ko 2023-06-23T00:52:19.712 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/ib700wdt.ko 2023-06-23T00:52:19.712 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/ibmasr.ko 2023-06-23T00:52:19.712 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-06-23T00:52:19.712 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/it8712f_wdt.ko 2023-06-23T00:52:19.713 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/it87_wdt.ko 2023-06-23T00:52:19.713 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/machzwd.ko 2023-06-23T00:52:19.713 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/nv_tco.ko 2023-06-23T00:52:19.713 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/pcwd_pci.ko 2023-06-23T00:52:19.713 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/pcwd_usb.ko 2023-06-23T00:52:19.713 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-06-23T00:52:19.713 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/sch311x_wdt.ko 2023-06-23T00:52:19.713 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/softdog.ko 2023-06-23T00:52:19.714 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/sp5100_tco.ko 2023-06-23T00:52:19.714 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/via_wdt.ko 2023-06-23T00:52:19.714 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-06-23T00:52:19.714 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/w83877f_wdt.ko 2023-06-23T00:52:19.714 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/w83977f_wdt.ko 2023-06-23T00:52:19.714 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/wdt_pci.ko 2023-06-23T00:52:19.714 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/xen_wdt.ko 2023-06-23T00:52:19.714 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/xen 2023-06-23T00:52:19.715 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/xen/xen-acpi-processor.ko 2023-06-23T00:52:19.715 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/xen/xen-evtchn.ko 2023-06-23T00:52:19.715 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/xen/xen-gntalloc.ko 2023-06-23T00:52:19.715 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/xen/xen-gntdev.ko 2023-06-23T00:52:19.715 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/xen/xen-pciback 2023-06-23T00:52:19.715 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-06-23T00:52:19.715 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/xen/xen-privcmd.ko 2023-06-23T00:52:19.715 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/xen/xenfs 2023-06-23T00:52:19.716 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/xen/xenfs/xenfs.ko 2023-06-23T00:52:19.716 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs 2023-06-23T00:52:19.716 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/9p 2023-06-23T00:52:19.716 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/9p/9p.ko 2023-06-23T00:52:19.716 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/affs 2023-06-23T00:52:19.716 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/affs/affs.ko 2023-06-23T00:52:19.716 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/befs 2023-06-23T00:52:19.717 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/befs/befs.ko 2023-06-23T00:52:19.717 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/binfmt_misc.ko 2023-06-23T00:52:19.717 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/btrfs 2023-06-23T00:52:19.717 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/btrfs/btrfs.ko 2023-06-23T00:52:19.717 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/cachefiles 2023-06-23T00:52:19.717 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/cachefiles/cachefiles.ko 2023-06-23T00:52:19.717 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/ceph 2023-06-23T00:52:19.717 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/ceph/ceph.ko 2023-06-23T00:52:19.718 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/coda 2023-06-23T00:52:19.718 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/coda/coda.ko 2023-06-23T00:52:19.718 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/cramfs 2023-06-23T00:52:19.718 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/cramfs/cramfs.ko 2023-06-23T00:52:19.718 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/dlm 2023-06-23T00:52:19.718 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/dlm/dlm.ko 2023-06-23T00:52:19.718 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/ecryptfs 2023-06-23T00:52:19.718 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/ecryptfs/ecryptfs.ko 2023-06-23T00:52:19.719 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/f2fs 2023-06-23T00:52:19.719 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/f2fs/f2fs.ko 2023-06-23T00:52:19.719 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/fat 2023-06-23T00:52:19.719 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/fat/fat.ko 2023-06-23T00:52:19.719 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/fat/msdos.ko 2023-06-23T00:52:19.719 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/fat/vfat.ko 2023-06-23T00:52:19.719 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/fscache 2023-06-23T00:52:19.719 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/fscache/fscache.ko 2023-06-23T00:52:19.720 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/fuse 2023-06-23T00:52:19.720 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/fuse/cuse.ko 2023-06-23T00:52:19.720 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/fuse/fuse.ko 2023-06-23T00:52:19.720 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/gfs2 2023-06-23T00:52:19.720 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/gfs2/gfs2.ko 2023-06-23T00:52:19.720 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/hfs 2023-06-23T00:52:19.720 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/hfs/hfs.ko 2023-06-23T00:52:19.720 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/hfsplus 2023-06-23T00:52:19.721 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/hfsplus/hfsplus.ko 2023-06-23T00:52:19.721 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/isofs 2023-06-23T00:52:19.721 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/isofs/isofs.ko 2023-06-23T00:52:19.721 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/jfs 2023-06-23T00:52:19.721 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/jfs/jfs.ko 2023-06-23T00:52:19.721 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/lockd 2023-06-23T00:52:19.721 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/lockd/lockd.ko 2023-06-23T00:52:19.721 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/minix 2023-06-23T00:52:19.722 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/minix/minix.ko 2023-06-23T00:52:19.722 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/netfs 2023-06-23T00:52:19.722 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/netfs/netfs.ko 2023-06-23T00:52:19.722 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nfs 2023-06-23T00:52:19.722 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nfs/blocklayout 2023-06-23T00:52:19.722 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-06-23T00:52:19.722 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nfs/filelayout 2023-06-23T00:52:19.723 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-06-23T00:52:19.723 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nfs/flexfilelayout 2023-06-23T00:52:19.723 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-06-23T00:52:19.723 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nfs/nfs.ko 2023-06-23T00:52:19.723 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nfs/nfsv3.ko 2023-06-23T00:52:19.723 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nfs/nfsv4.ko 2023-06-23T00:52:19.723 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nfs_common 2023-06-23T00:52:19.724 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nfs_common/grace.ko 2023-06-23T00:52:19.725 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nfs_common/nfs_acl.ko 2023-06-23T00:52:19.725 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nfsd 2023-06-23T00:52:19.725 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nfsd/nfsd.ko 2023-06-23T00:52:19.725 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nilfs2 2023-06-23T00:52:19.726 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nilfs2/nilfs2.ko 2023-06-23T00:52:19.726 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls 2023-06-23T00:52:19.726 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/mac-celtic.ko 2023-06-23T00:52:19.726 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/mac-centeuro.ko 2023-06-23T00:52:19.727 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/mac-croatian.ko 2023-06-23T00:52:19.727 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/mac-cyrillic.ko 2023-06-23T00:52:19.727 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/mac-gaelic.ko 2023-06-23T00:52:19.727 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/mac-greek.ko 2023-06-23T00:52:19.728 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/mac-iceland.ko 2023-06-23T00:52:19.728 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/mac-inuit.ko 2023-06-23T00:52:19.728 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/mac-roman.ko 2023-06-23T00:52:19.729 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/mac-romanian.ko 2023-06-23T00:52:19.729 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/mac-turkish.ko 2023-06-23T00:52:19.729 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp1250.ko 2023-06-23T00:52:19.729 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp1251.ko 2023-06-23T00:52:19.729 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp1255.ko 2023-06-23T00:52:19.729 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp737.ko 2023-06-23T00:52:19.729 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp775.ko 2023-06-23T00:52:19.729 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp850.ko 2023-06-23T00:52:19.730 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp852.ko 2023-06-23T00:52:19.730 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp855.ko 2023-06-23T00:52:19.730 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp857.ko 2023-06-23T00:52:19.730 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp860.ko 2023-06-23T00:52:19.730 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp861.ko 2023-06-23T00:52:19.730 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp862.ko 2023-06-23T00:52:19.730 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp863.ko 2023-06-23T00:52:19.731 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp864.ko 2023-06-23T00:52:19.731 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp865.ko 2023-06-23T00:52:19.731 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp866.ko 2023-06-23T00:52:19.731 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp869.ko 2023-06-23T00:52:19.731 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp874.ko 2023-06-23T00:52:19.731 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp932.ko 2023-06-23T00:52:19.731 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp936.ko 2023-06-23T00:52:19.731 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp949.ko 2023-06-23T00:52:19.732 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp950.ko 2023-06-23T00:52:19.732 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_euc-jp.ko 2023-06-23T00:52:19.732 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_iso8859-1.ko 2023-06-23T00:52:19.732 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_iso8859-13.ko 2023-06-23T00:52:19.732 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_iso8859-14.ko 2023-06-23T00:52:19.732 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_iso8859-15.ko 2023-06-23T00:52:19.733 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_iso8859-2.ko 2023-06-23T00:52:19.733 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_iso8859-3.ko 2023-06-23T00:52:19.733 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_iso8859-4.ko 2023-06-23T00:52:19.733 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_iso8859-5.ko 2023-06-23T00:52:19.734 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_iso8859-6.ko 2023-06-23T00:52:19.734 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_iso8859-7.ko 2023-06-23T00:52:19.734 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_iso8859-9.ko 2023-06-23T00:52:19.734 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_koi8-r.ko 2023-06-23T00:52:19.734 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_koi8-ru.ko 2023-06-23T00:52:19.735 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_koi8-u.ko 2023-06-23T00:52:19.735 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_utf8.ko 2023-06-23T00:52:19.735 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/ocfs2 2023-06-23T00:52:19.735 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/ocfs2/cluster 2023-06-23T00:52:19.736 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-06-23T00:52:19.736 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/ocfs2/dlm 2023-06-23T00:52:19.736 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-06-23T00:52:19.737 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/ocfs2/dlmfs 2023-06-23T00:52:19.737 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-06-23T00:52:19.737 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/ocfs2/ocfs2.ko 2023-06-23T00:52:19.738 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-06-23T00:52:19.738 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-06-23T00:52:19.738 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-06-23T00:52:19.738 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/overlayfs 2023-06-23T00:52:19.738 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/overlayfs/overlay.ko 2023-06-23T00:52:19.739 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/pstore 2023-06-23T00:52:19.739 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/pstore/ramoops.ko 2023-06-23T00:52:19.739 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/reiserfs 2023-06-23T00:52:19.739 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/reiserfs/reiserfs.ko 2023-06-23T00:52:19.740 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/romfs 2023-06-23T00:52:19.740 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/romfs/romfs.ko 2023-06-23T00:52:19.740 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/smb 2023-06-23T00:52:19.741 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/smb/client 2023-06-23T00:52:19.741 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/smb/client/cifs.ko 2023-06-23T00:52:19.741 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/smb/common 2023-06-23T00:52:19.741 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/smb/common/cifs_arc4.ko 2023-06-23T00:52:19.742 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/smb/common/cifs_md4.ko 2023-06-23T00:52:19.742 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/squashfs 2023-06-23T00:52:19.742 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/squashfs/squashfs.ko 2023-06-23T00:52:19.742 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/sysv 2023-06-23T00:52:19.742 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/sysv/sysv.ko 2023-06-23T00:52:19.742 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/udf 2023-06-23T00:52:19.742 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/udf/udf.ko 2023-06-23T00:52:19.742 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/ufs 2023-06-23T00:52:19.743 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/ufs/ufs.ko 2023-06-23T00:52:19.743 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/xfs 2023-06-23T00:52:19.743 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/xfs/xfs.ko 2023-06-23T00:52:19.743 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/kernel 2023-06-23T00:52:19.743 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/kernel/rcu 2023-06-23T00:52:19.744 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/kernel/rcu/rcutorture.ko 2023-06-23T00:52:19.744 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/kernel/torture.ko 2023-06-23T00:52:19.744 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/kernel/trace 2023-06-23T00:52:19.745 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-06-23T00:52:19.745 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib 2023-06-23T00:52:19.745 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/asn1_encoder.ko 2023-06-23T00:52:19.745 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/crc-itu-t.ko 2023-06-23T00:52:19.746 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/crc8.ko 2023-06-23T00:52:19.746 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/crypto 2023-06-23T00:52:19.746 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/crypto/libarc4.ko 2023-06-23T00:52:19.746 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/crypto/libdes.ko 2023-06-23T00:52:19.747 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/libcrc32c.ko 2023-06-23T00:52:19.747 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/lru_cache.ko 2023-06-23T00:52:19.747 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/lz4 2023-06-23T00:52:19.748 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/lz4/lz4_compress.ko 2023-06-23T00:52:19.748 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/lz4/lz4hc_compress.ko 2023-06-23T00:52:19.748 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/math 2023-06-23T00:52:19.748 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/math/cordic.ko 2023-06-23T00:52:19.749 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/raid6 2023-06-23T00:52:19.749 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/raid6/raid6_pq.ko 2023-06-23T00:52:19.749 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/reed_solomon 2023-06-23T00:52:19.750 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/reed_solomon/reed_solomon.ko 2023-06-23T00:52:19.750 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/ts_bm.ko 2023-06-23T00:52:19.750 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/ts_fsm.ko 2023-06-23T00:52:19.750 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/ts_kmp.ko 2023-06-23T00:52:19.751 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/zstd 2023-06-23T00:52:19.751 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/zstd/zstd_compress.ko 2023-06-23T00:52:19.751 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/mm 2023-06-23T00:52:19.752 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/mm/hwpoison-inject.ko 2023-06-23T00:52:19.752 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net 2023-06-23T00:52:19.752 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/6lowpan 2023-06-23T00:52:19.752 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/6lowpan/6lowpan.ko 2023-06-23T00:52:19.753 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/6lowpan/nhc_dest.ko 2023-06-23T00:52:19.753 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/6lowpan/nhc_fragment.ko 2023-06-23T00:52:19.753 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/6lowpan/nhc_hop.ko 2023-06-23T00:52:19.753 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/6lowpan/nhc_ipv6.ko 2023-06-23T00:52:19.753 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/6lowpan/nhc_mobility.ko 2023-06-23T00:52:19.754 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/6lowpan/nhc_routing.ko 2023-06-23T00:52:19.754 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/6lowpan/nhc_udp.ko 2023-06-23T00:52:19.754 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/802 2023-06-23T00:52:19.754 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/802/garp.ko 2023-06-23T00:52:19.755 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/802/mrp.ko 2023-06-23T00:52:19.755 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/802/p8022.ko 2023-06-23T00:52:19.755 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/802/psnap.ko 2023-06-23T00:52:19.756 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/802/stp.ko 2023-06-23T00:52:19.756 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/8021q 2023-06-23T00:52:19.756 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/8021q/8021q.ko 2023-06-23T00:52:19.756 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/9p 2023-06-23T00:52:19.757 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/9p/9pnet.ko 2023-06-23T00:52:19.757 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/9p/9pnet_fd.ko 2023-06-23T00:52:19.757 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/9p/9pnet_rdma.ko 2023-06-23T00:52:19.757 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/9p/9pnet_virtio.ko 2023-06-23T00:52:19.758 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/batman-adv 2023-06-23T00:52:19.758 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/batman-adv/batman-adv.ko 2023-06-23T00:52:19.758 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge 2023-06-23T00:52:19.758 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/br_netfilter.ko 2023-06-23T00:52:19.759 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/bridge.ko 2023-06-23T00:52:19.759 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter 2023-06-23T00:52:19.759 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-06-23T00:52:19.760 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebt_among.ko 2023-06-23T00:52:19.760 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebt_arp.ko 2023-06-23T00:52:19.760 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-06-23T00:52:19.761 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-06-23T00:52:19.761 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebt_ip.ko 2023-06-23T00:52:19.761 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-06-23T00:52:19.762 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebt_limit.ko 2023-06-23T00:52:19.762 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebt_log.ko 2023-06-23T00:52:19.762 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebt_mark.ko 2023-06-23T00:52:19.762 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-06-23T00:52:19.763 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-06-23T00:52:19.763 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-06-23T00:52:19.763 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-06-23T00:52:19.764 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebt_snat.ko 2023-06-23T00:52:19.764 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebt_stp.ko 2023-06-23T00:52:19.764 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-06-23T00:52:19.764 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-06-23T00:52:19.765 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-06-23T00:52:19.765 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-06-23T00:52:19.765 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebtables.ko 2023-06-23T00:52:19.765 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-06-23T00:52:19.765 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-06-23T00:52:19.766 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-06-23T00:52:19.766 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ceph 2023-06-23T00:52:19.766 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ceph/libceph.ko 2023-06-23T00:52:19.766 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/core 2023-06-23T00:52:19.767 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/core/failover.ko 2023-06-23T00:52:19.767 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/core/pktgen.ko 2023-06-23T00:52:19.767 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/dccp 2023-06-23T00:52:19.767 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/dccp/dccp.ko 2023-06-23T00:52:19.768 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/dccp/dccp_diag.ko 2023-06-23T00:52:19.768 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/dccp/dccp_ipv4.ko 2023-06-23T00:52:19.768 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/dccp/dccp_ipv6.ko 2023-06-23T00:52:19.768 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/dns_resolver 2023-06-23T00:52:19.768 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/dns_resolver/dns_resolver.ko 2023-06-23T00:52:19.768 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/dsa 2023-06-23T00:52:19.769 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/dsa/dsa_core.ko 2023-06-23T00:52:19.769 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/dsa/tag_brcm.ko 2023-06-23T00:52:19.769 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/dsa/tag_dsa.ko 2023-06-23T00:52:19.769 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/dsa/tag_none.ko 2023-06-23T00:52:19.769 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/dsa/tag_trailer.ko 2023-06-23T00:52:19.769 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ieee802154 2023-06-23T00:52:19.769 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ieee802154/6lowpan 2023-06-23T00:52:19.770 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-06-23T00:52:19.770 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ieee802154/ieee802154.ko 2023-06-23T00:52:19.770 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ieee802154/ieee802154_socket.ko 2023-06-23T00:52:19.770 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4 2023-06-23T00:52:19.771 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/ah4.ko 2023-06-23T00:52:19.771 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/esp4.ko 2023-06-23T00:52:19.771 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/fou.ko 2023-06-23T00:52:19.771 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/gre.ko 2023-06-23T00:52:19.772 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/inet_diag.ko 2023-06-23T00:52:19.772 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/ip_gre.ko 2023-06-23T00:52:19.772 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/ip_tunnel.ko 2023-06-23T00:52:19.773 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/ip_vti.ko 2023-06-23T00:52:19.773 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/ipcomp.ko 2023-06-23T00:52:19.773 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/ipip.ko 2023-06-23T00:52:19.773 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter 2023-06-23T00:52:19.774 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/arp_tables.ko 2023-06-23T00:52:19.774 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-06-23T00:52:19.774 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-06-23T00:52:19.774 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-06-23T00:52:19.775 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-06-23T00:52:19.775 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-06-23T00:52:19.775 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-06-23T00:52:19.776 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-06-23T00:52:19.776 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-06-23T00:52:19.776 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-06-23T00:52:19.777 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/iptable_security.ko 2023-06-23T00:52:19.777 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-06-23T00:52:19.777 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-06-23T00:52:19.777 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-06-23T00:52:19.778 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-06-23T00:52:19.778 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-06-23T00:52:19.778 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-06-23T00:52:19.778 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-06-23T00:52:19.779 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-06-23T00:52:19.779 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-06-23T00:52:19.779 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-06-23T00:52:19.779 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/tcp_bic.ko 2023-06-23T00:52:19.779 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/tcp_dctcp.ko 2023-06-23T00:52:19.780 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/tcp_diag.ko 2023-06-23T00:52:19.780 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/tcp_highspeed.ko 2023-06-23T00:52:19.780 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/tcp_htcp.ko 2023-06-23T00:52:19.781 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/tcp_hybla.ko 2023-06-23T00:52:19.781 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/tcp_illinois.ko 2023-06-23T00:52:19.781 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/tcp_lp.ko 2023-06-23T00:52:19.781 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/tcp_scalable.ko 2023-06-23T00:52:19.781 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/tcp_vegas.ko 2023-06-23T00:52:19.782 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/tcp_veno.ko 2023-06-23T00:52:19.782 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/tcp_westwood.ko 2023-06-23T00:52:19.782 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/tcp_yeah.ko 2023-06-23T00:52:19.782 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/tunnel4.ko 2023-06-23T00:52:19.782 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/udp_diag.ko 2023-06-23T00:52:19.782 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/udp_tunnel.ko 2023-06-23T00:52:19.783 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/xfrm4_tunnel.ko 2023-06-23T00:52:19.783 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6 2023-06-23T00:52:19.784 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/ah6.ko 2023-06-23T00:52:19.784 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/esp6.ko 2023-06-23T00:52:19.784 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/fou6.ko 2023-06-23T00:52:19.784 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/ip6_tunnel.ko 2023-06-23T00:52:19.785 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-06-23T00:52:19.785 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/ip6_vti.ko 2023-06-23T00:52:19.785 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/ipcomp6.ko 2023-06-23T00:52:19.785 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter 2023-06-23T00:52:19.785 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-06-23T00:52:19.785 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-06-23T00:52:19.785 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-06-23T00:52:19.785 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-06-23T00:52:19.786 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-06-23T00:52:19.786 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-06-23T00:52:19.786 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-06-23T00:52:19.786 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-06-23T00:52:19.786 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-06-23T00:52:19.786 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-06-23T00:52:19.786 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-06-23T00:52:19.786 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-06-23T00:52:19.787 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-06-23T00:52:19.787 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-06-23T00:52:19.787 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-06-23T00:52:19.787 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-06-23T00:52:19.787 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-06-23T00:52:19.787 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-06-23T00:52:19.787 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-06-23T00:52:19.787 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-06-23T00:52:19.788 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-06-23T00:52:19.788 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-06-23T00:52:19.788 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-06-23T00:52:19.788 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-06-23T00:52:19.788 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/sit.ko 2023-06-23T00:52:19.788 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/tunnel6.ko 2023-06-23T00:52:19.788 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/xfrm6_tunnel.ko 2023-06-23T00:52:19.789 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/key 2023-06-23T00:52:19.789 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/key/af_key.ko 2023-06-23T00:52:19.789 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/l2tp 2023-06-23T00:52:19.789 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/l2tp/l2tp_core.ko 2023-06-23T00:52:19.789 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/l2tp/l2tp_debugfs.ko 2023-06-23T00:52:19.789 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/l2tp/l2tp_eth.ko 2023-06-23T00:52:19.789 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/l2tp/l2tp_ip.ko 2023-06-23T00:52:19.789 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/l2tp/l2tp_ip6.ko 2023-06-23T00:52:19.790 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/l2tp/l2tp_netlink.ko 2023-06-23T00:52:19.790 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/l2tp/l2tp_ppp.ko 2023-06-23T00:52:19.790 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/llc 2023-06-23T00:52:19.790 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/llc/llc.ko 2023-06-23T00:52:19.790 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/mac802154 2023-06-23T00:52:19.790 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/mac802154/mac802154.ko 2023-06-23T00:52:19.790 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/mpls 2023-06-23T00:52:19.790 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/mpls/mpls_gso.ko 2023-06-23T00:52:19.791 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter 2023-06-23T00:52:19.791 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipset 2023-06-23T00:52:19.791 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipset/ip_set.ko 2023-06-23T00:52:19.791 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-06-23T00:52:19.791 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-06-23T00:52:19.791 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-06-23T00:52:19.791 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-06-23T00:52:19.791 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-06-23T00:52:19.792 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-06-23T00:52:19.792 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-06-23T00:52:19.792 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-06-23T00:52:19.792 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-06-23T00:52:19.792 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-06-23T00:52:19.792 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-06-23T00:52:19.792 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-06-23T00:52:19.793 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-06-23T00:52:19.793 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-06-23T00:52:19.793 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-06-23T00:52:19.793 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipvs 2023-06-23T00:52:19.793 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipvs/ip_vs.ko 2023-06-23T00:52:19.793 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-06-23T00:52:19.793 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-06-23T00:52:19.793 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-06-23T00:52:19.794 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-06-23T00:52:19.794 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-06-23T00:52:19.794 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-06-23T00:52:19.794 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-06-23T00:52:19.794 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-06-23T00:52:19.794 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-06-23T00:52:19.794 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-06-23T00:52:19.794 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-06-23T00:52:19.795 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-06-23T00:52:19.795 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-06-23T00:52:19.795 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_conncount.ko 2023-06-23T00:52:19.795 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_conntrack.ko 2023-06-23T00:52:19.795 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-06-23T00:52:19.795 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-06-23T00:52:19.795 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-06-23T00:52:19.796 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_conntrack_h323.ko 2023-06-23T00:52:19.796 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_conntrack_irc.ko 2023-06-23T00:52:19.796 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-06-23T00:52:19.796 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-06-23T00:52:19.796 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-06-23T00:52:19.796 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_conntrack_sane.ko 2023-06-23T00:52:19.796 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_conntrack_sip.ko 2023-06-23T00:52:19.796 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-06-23T00:52:19.797 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-06-23T00:52:19.797 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_dup_netdev.ko 2023-06-23T00:52:19.797 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_flow_table.ko 2023-06-23T00:52:19.797 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_flow_table_inet.ko 2023-06-23T00:52:19.797 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_log_syslog.ko 2023-06-23T00:52:19.797 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_nat.ko 2023-06-23T00:52:19.797 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_nat_amanda.ko 2023-06-23T00:52:19.797 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_nat_ftp.ko 2023-06-23T00:52:19.798 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_nat_irc.ko 2023-06-23T00:52:19.798 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_nat_sip.ko 2023-06-23T00:52:19.798 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_nat_tftp.ko 2023-06-23T00:52:19.798 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_synproxy_core.ko 2023-06-23T00:52:19.798 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_tables.ko 2023-06-23T00:52:19.798 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nfnetlink.ko 2023-06-23T00:52:19.798 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nfnetlink_acct.ko 2023-06-23T00:52:19.798 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nfnetlink_log.ko 2023-06-23T00:52:19.799 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nfnetlink_osf.ko 2023-06-23T00:52:19.799 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nfnetlink_queue.ko 2023-06-23T00:52:19.799 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nft_chain_nat.ko 2023-06-23T00:52:19.799 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nft_compat.ko 2023-06-23T00:52:19.799 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nft_ct.ko 2023-06-23T00:52:19.799 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nft_dup_netdev.ko 2023-06-23T00:52:19.799 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nft_fib.ko 2023-06-23T00:52:19.799 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nft_fwd_netdev.ko 2023-06-23T00:52:19.800 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nft_hash.ko 2023-06-23T00:52:19.800 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nft_limit.ko 2023-06-23T00:52:19.800 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nft_log.ko 2023-06-23T00:52:19.800 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nft_masq.ko 2023-06-23T00:52:19.800 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nft_nat.ko 2023-06-23T00:52:19.800 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nft_numgen.ko 2023-06-23T00:52:19.800 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nft_queue.ko 2023-06-23T00:52:19.801 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nft_quota.ko 2023-06-23T00:52:19.801 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nft_redir.ko 2023-06-23T00:52:19.801 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nft_reject.ko 2023-06-23T00:52:19.801 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nft_reject_inet.ko 2023-06-23T00:52:19.801 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nft_socket.ko 2023-06-23T00:52:19.801 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nft_tunnel.ko 2023-06-23T00:52:19.801 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_AUDIT.ko 2023-06-23T00:52:19.801 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_CHECKSUM.ko 2023-06-23T00:52:19.802 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_CLASSIFY.ko 2023-06-23T00:52:19.802 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-06-23T00:52:19.802 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_CT.ko 2023-06-23T00:52:19.802 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_DSCP.ko 2023-06-23T00:52:19.802 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_HL.ko 2023-06-23T00:52:19.802 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_HMARK.ko 2023-06-23T00:52:19.802 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_IDLETIMER.ko 2023-06-23T00:52:19.802 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_LED.ko 2023-06-23T00:52:19.803 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_LOG.ko 2023-06-23T00:52:19.803 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_MASQUERADE.ko 2023-06-23T00:52:19.803 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_NETMAP.ko 2023-06-23T00:52:19.803 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_NFLOG.ko 2023-06-23T00:52:19.803 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_NFQUEUE.ko 2023-06-23T00:52:19.803 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_RATEEST.ko 2023-06-23T00:52:19.803 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_REDIRECT.ko 2023-06-23T00:52:19.803 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_SECMARK.ko 2023-06-23T00:52:19.804 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_TCPMSS.ko 2023-06-23T00:52:19.804 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-06-23T00:52:19.804 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_TEE.ko 2023-06-23T00:52:19.804 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_TPROXY.ko 2023-06-23T00:52:19.804 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_TRACE.ko 2023-06-23T00:52:19.804 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_addrtype.ko 2023-06-23T00:52:19.804 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_bpf.ko 2023-06-23T00:52:19.804 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_cgroup.ko 2023-06-23T00:52:19.805 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_cluster.ko 2023-06-23T00:52:19.805 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_comment.ko 2023-06-23T00:52:19.805 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_connbytes.ko 2023-06-23T00:52:19.805 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_connlabel.ko 2023-06-23T00:52:19.805 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_connlimit.ko 2023-06-23T00:52:19.805 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_connmark.ko 2023-06-23T00:52:19.805 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_conntrack.ko 2023-06-23T00:52:19.805 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_cpu.ko 2023-06-23T00:52:19.806 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_dccp.ko 2023-06-23T00:52:19.806 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_devgroup.ko 2023-06-23T00:52:19.806 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_dscp.ko 2023-06-23T00:52:19.806 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_ecn.ko 2023-06-23T00:52:19.806 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_esp.ko 2023-06-23T00:52:19.806 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_hashlimit.ko 2023-06-23T00:52:19.806 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_helper.ko 2023-06-23T00:52:19.806 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_hl.ko 2023-06-23T00:52:19.807 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_ipcomp.ko 2023-06-23T00:52:19.807 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_iprange.ko 2023-06-23T00:52:19.807 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_ipvs.ko 2023-06-23T00:52:19.807 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_l2tp.ko 2023-06-23T00:52:19.807 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_length.ko 2023-06-23T00:52:19.807 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_limit.ko 2023-06-23T00:52:19.807 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_mac.ko 2023-06-23T00:52:19.808 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_mark.ko 2023-06-23T00:52:19.808 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_multiport.ko 2023-06-23T00:52:19.808 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_nat.ko 2023-06-23T00:52:19.808 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_nfacct.ko 2023-06-23T00:52:19.808 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_osf.ko 2023-06-23T00:52:19.808 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_owner.ko 2023-06-23T00:52:19.808 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_physdev.ko 2023-06-23T00:52:19.808 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_pkttype.ko 2023-06-23T00:52:19.809 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_policy.ko 2023-06-23T00:52:19.809 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_quota.ko 2023-06-23T00:52:19.809 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_rateest.ko 2023-06-23T00:52:19.809 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_realm.ko 2023-06-23T00:52:19.809 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_recent.ko 2023-06-23T00:52:19.809 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_sctp.ko 2023-06-23T00:52:19.809 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_set.ko 2023-06-23T00:52:19.809 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_socket.ko 2023-06-23T00:52:19.810 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_state.ko 2023-06-23T00:52:19.810 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_statistic.ko 2023-06-23T00:52:19.810 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_string.ko 2023-06-23T00:52:19.810 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_tcpmss.ko 2023-06-23T00:52:19.810 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_time.ko 2023-06-23T00:52:19.810 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_u32.ko 2023-06-23T00:52:19.810 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netlink 2023-06-23T00:52:19.810 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netlink/netlink_diag.ko 2023-06-23T00:52:19.811 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/nsh 2023-06-23T00:52:19.811 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/nsh/nsh.ko 2023-06-23T00:52:19.811 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/openvswitch 2023-06-23T00:52:19.812 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/openvswitch/openvswitch.ko 2023-06-23T00:52:19.812 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/openvswitch/vport-geneve.ko 2023-06-23T00:52:19.812 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/openvswitch/vport-gre.ko 2023-06-23T00:52:19.812 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/openvswitch/vport-vxlan.ko 2023-06-23T00:52:19.813 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/packet 2023-06-23T00:52:19.813 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/packet/af_packet_diag.ko 2023-06-23T00:52:19.813 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/rds 2023-06-23T00:52:19.813 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/rds/rds.ko 2023-06-23T00:52:19.813 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/rds/rds_rdma.ko 2023-06-23T00:52:19.813 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/rds/rds_tcp.ko 2023-06-23T00:52:19.813 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched 2023-06-23T00:52:19.814 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/act_csum.ko 2023-06-23T00:52:19.814 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/act_gact.ko 2023-06-23T00:52:19.814 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/act_ipt.ko 2023-06-23T00:52:19.814 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/act_mirred.ko 2023-06-23T00:52:19.814 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/act_nat.ko 2023-06-23T00:52:19.814 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/act_pedit.ko 2023-06-23T00:52:19.814 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/act_police.ko 2023-06-23T00:52:19.815 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/act_simple.ko 2023-06-23T00:52:19.815 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/act_skbedit.ko 2023-06-23T00:52:19.815 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/act_vlan.ko 2023-06-23T00:52:19.815 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/cls_basic.ko 2023-06-23T00:52:19.815 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/cls_bpf.ko 2023-06-23T00:52:19.816 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/cls_flow.ko 2023-06-23T00:52:19.816 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/cls_fw.ko 2023-06-23T00:52:19.816 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/cls_route.ko 2023-06-23T00:52:19.816 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/cls_u32.ko 2023-06-23T00:52:19.816 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/em_cmp.ko 2023-06-23T00:52:19.816 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/em_ipset.ko 2023-06-23T00:52:19.817 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/em_meta.ko 2023-06-23T00:52:19.817 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/em_nbyte.ko 2023-06-23T00:52:19.817 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/em_text.ko 2023-06-23T00:52:19.817 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/em_u32.ko 2023-06-23T00:52:19.817 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_choke.ko 2023-06-23T00:52:19.818 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_codel.ko 2023-06-23T00:52:19.818 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_drr.ko 2023-06-23T00:52:19.818 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_fq.ko 2023-06-23T00:52:19.818 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_gred.ko 2023-06-23T00:52:19.818 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_hfsc.ko 2023-06-23T00:52:19.818 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_hhf.ko 2023-06-23T00:52:19.818 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_htb.ko 2023-06-23T00:52:19.818 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_ingress.ko 2023-06-23T00:52:19.819 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_mqprio.ko 2023-06-23T00:52:19.819 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_mqprio_lib.ko 2023-06-23T00:52:19.819 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_multiq.ko 2023-06-23T00:52:19.819 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_netem.ko 2023-06-23T00:52:19.819 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_pie.ko 2023-06-23T00:52:19.819 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_plug.ko 2023-06-23T00:52:19.819 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_prio.ko 2023-06-23T00:52:19.820 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_qfq.ko 2023-06-23T00:52:19.820 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_red.ko 2023-06-23T00:52:19.820 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_sfb.ko 2023-06-23T00:52:19.820 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_sfq.ko 2023-06-23T00:52:19.820 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_tbf.ko 2023-06-23T00:52:19.821 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_teql.ko 2023-06-23T00:52:19.821 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sctp 2023-06-23T00:52:19.821 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sctp/sctp.ko 2023-06-23T00:52:19.821 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sctp/sctp_diag.ko 2023-06-23T00:52:19.821 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sunrpc 2023-06-23T00:52:19.821 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sunrpc/auth_gss 2023-06-23T00:52:19.822 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-06-23T00:52:19.822 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-06-23T00:52:19.822 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sunrpc/sunrpc.ko 2023-06-23T00:52:19.822 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sunrpc/xprtrdma 2023-06-23T00:52:19.822 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-06-23T00:52:19.822 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/tipc 2023-06-23T00:52:19.822 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/tipc/diag.ko 2023-06-23T00:52:19.823 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/tipc/tipc.ko 2023-06-23T00:52:19.823 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/unix 2023-06-23T00:52:19.823 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/unix/unix_diag.ko 2023-06-23T00:52:19.823 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/vmw_vsock 2023-06-23T00:52:19.823 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-06-23T00:52:19.823 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-06-23T00:52:19.824 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/vmw_vsock/vsock.ko 2023-06-23T00:52:19.824 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/vmw_vsock/vsock_diag.ko 2023-06-23T00:52:19.824 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/vmw_vsock/vsock_loopback.ko 2023-06-23T00:52:19.824 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/xfrm 2023-06-23T00:52:19.824 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/xfrm/xfrm_ipcomp.ko 2023-06-23T00:52:19.824 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/security 2023-06-23T00:52:19.824 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/security/keys 2023-06-23T00:52:19.825 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/security/keys/encrypted-keys 2023-06-23T00:52:19.825 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-06-23T00:52:19.825 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/security/keys/trusted-keys 2023-06-23T00:52:19.825 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/security/keys/trusted-keys/trusted.ko 2023-06-23T00:52:19.825 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/virt 2023-06-23T00:52:19.825 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/virt/lib 2023-06-23T00:52:19.825 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/virt/lib/irqbypass.ko 2023-06-23T00:52:19.826 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/modules.alias 2023-06-23T00:52:19.826 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/modules.alias.bin 2023-06-23T00:52:19.826 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/modules.builtin 2023-06-23T00:52:19.826 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/modules.builtin.bin 2023-06-23T00:52:19.826 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/modules.builtin.modinfo 2023-06-23T00:52:19.826 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/modules.dep 2023-06-23T00:52:19.826 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/modules.dep.bin 2023-06-23T00:52:19.827 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/modules.devname 2023-06-23T00:52:19.827 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/modules.order 2023-06-23T00:52:19.827 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/modules.softdep 2023-06-23T00:52:19.827 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/modules.symbols 2023-06-23T00:52:19.827 INFO:teuthology.orchestra.run.smithi120.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/modules.symbols.bin 2023-06-23T00:52:19.828 DEBUG:teuthology.task.kernel:get_image_version: 6.4.0-rc5-g1c622d92b6f1 2023-06-23T00:52:19.828 DEBUG:teuthology.orchestra.run.smithi120:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-06-23T00:52:19.849 INFO:teuthology.orchestra.run.smithi120.stdout:postinstall scriptlet (using /bin/sh): 2023-06-23T00:52:19.850 INFO:teuthology.orchestra.run.smithi120.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.4.0-rc5-g1c622d92b6f1 -a -r /boot/System.map-6.4.0-rc5-g1c622d92b6f1 ]; then 2023-06-23T00:52:19.850 INFO:teuthology.orchestra.run.smithi120.stdout:cp /boot/vmlinuz-6.4.0-rc5-g1c622d92b6f1 /boot/.vmlinuz-6.4.0-rc5-g1c622d92b6f1-rpm 2023-06-23T00:52:19.850 INFO:teuthology.orchestra.run.smithi120.stdout:cp /boot/System.map-6.4.0-rc5-g1c622d92b6f1 /boot/.System.map-6.4.0-rc5-g1c622d92b6f1-rpm 2023-06-23T00:52:19.850 INFO:teuthology.orchestra.run.smithi120.stdout:rm -f /boot/vmlinuz-6.4.0-rc5-g1c622d92b6f1 /boot/System.map-6.4.0-rc5-g1c622d92b6f1 2023-06-23T00:52:19.850 INFO:teuthology.orchestra.run.smithi120.stdout:/sbin/installkernel 6.4.0-rc5-g1c622d92b6f1 /boot/.vmlinuz-6.4.0-rc5-g1c622d92b6f1-rpm /boot/.System.map-6.4.0-rc5-g1c622d92b6f1-rpm 2023-06-23T00:52:19.850 INFO:teuthology.orchestra.run.smithi120.stdout:rm -f /boot/.vmlinuz-6.4.0-rc5-g1c622d92b6f1-rpm /boot/.System.map-6.4.0-rc5-g1c622d92b6f1-rpm 2023-06-23T00:52:19.851 INFO:teuthology.orchestra.run.smithi120.stdout:fi 2023-06-23T00:52:19.851 INFO:teuthology.orchestra.run.smithi120.stdout:preuninstall scriptlet (using /bin/sh): 2023-06-23T00:52:19.851 INFO:teuthology.orchestra.run.smithi120.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-06-23T00:52:19.851 INFO:teuthology.orchestra.run.smithi120.stdout:new-kernel-pkg --remove 6.4.0-rc5-g1c622d92b6f1 --rminitrd --initrdfile=/boot/initramfs-6.4.0-rc5-g1c622d92b6f1.img 2023-06-23T00:52:19.851 INFO:teuthology.orchestra.run.smithi120.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-06-23T00:52:19.851 INFO:teuthology.orchestra.run.smithi120.stdout:kernel-install remove 6.4.0-rc5-g1c622d92b6f1 2023-06-23T00:52:19.851 INFO:teuthology.orchestra.run.smithi120.stdout:fi 2023-06-23T00:52:19.852 INFO:teuthology.orchestra.run.smithi120.stdout:postuninstall scriptlet (using /bin/sh): 2023-06-23T00:52:19.852 INFO:teuthology.orchestra.run.smithi120.stdout:if [ -x /sbin/update-bootloader ]; then 2023-06-23T00:52:19.852 INFO:teuthology.orchestra.run.smithi120.stdout:/sbin/update-bootloader --remove 6.4.0-rc5-g1c622d92b6f1 2023-06-23T00:52:19.852 INFO:teuthology.orchestra.run.smithi120.stdout:fi 2023-06-23T00:52:19.853 DEBUG:teuthology.orchestra.run.smithi120:> sudo rpm -qi grub2-tools 2023-06-23T00:52:19.925 INFO:teuthology.orchestra.run.smithi120.stdout:Name : grub2-tools 2023-06-23T00:52:19.925 INFO:teuthology.orchestra.run.smithi120.stdout:Epoch : 1 2023-06-23T00:52:19.925 INFO:teuthology.orchestra.run.smithi120.stdout:Version : 2.02 2023-06-23T00:52:19.926 INFO:teuthology.orchestra.run.smithi120.stdout:Release : 99.el8_4.1 2023-06-23T00:52:19.926 INFO:teuthology.orchestra.run.smithi120.stdout:Architecture: x86_64 2023-06-23T00:52:19.926 INFO:teuthology.orchestra.run.smithi120.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-06-23T00:52:19.926 INFO:teuthology.orchestra.run.smithi120.stdout:Group : System Environment/Base 2023-06-23T00:52:19.926 INFO:teuthology.orchestra.run.smithi120.stdout:Size : 9264631 2023-06-23T00:52:19.926 INFO:teuthology.orchestra.run.smithi120.stdout:License : GPLv3+ 2023-06-23T00:52:19.926 INFO:teuthology.orchestra.run.smithi120.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-06-23T00:52:19.927 INFO:teuthology.orchestra.run.smithi120.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-06-23T00:52:19.927 INFO:teuthology.orchestra.run.smithi120.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-06-23T00:52:19.927 INFO:teuthology.orchestra.run.smithi120.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-06-23T00:52:19.927 INFO:teuthology.orchestra.run.smithi120.stdout:Relocations : (not relocatable) 2023-06-23T00:52:19.927 INFO:teuthology.orchestra.run.smithi120.stdout:Packager : Red Hat, Inc. 2023-06-23T00:52:19.927 INFO:teuthology.orchestra.run.smithi120.stdout:Vendor : Red Hat, Inc. 2023-06-23T00:52:19.927 INFO:teuthology.orchestra.run.smithi120.stdout:URL : http://www.gnu.org/software/grub/ 2023-06-23T00:52:19.928 INFO:teuthology.orchestra.run.smithi120.stdout:Summary : Support tools for GRUB. 2023-06-23T00:52:19.928 INFO:teuthology.orchestra.run.smithi120.stdout:Description : 2023-06-23T00:52:19.928 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T00:52:19.928 INFO:teuthology.orchestra.run.smithi120.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-06-23T00:52:19.928 INFO:teuthology.orchestra.run.smithi120.stdout:customizable bootloader with modular architecture. It supports a rich 2023-06-23T00:52:19.928 INFO:teuthology.orchestra.run.smithi120.stdout:variety of kernel formats, file systems, computer architectures and 2023-06-23T00:52:19.929 INFO:teuthology.orchestra.run.smithi120.stdout:hardware devices. 2023-06-23T00:52:19.929 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T00:52:19.929 INFO:teuthology.orchestra.run.smithi120.stdout:This subpackage provides tools for support of all platforms. 2023-06-23T00:52:19.930 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-06-23T00:52:19.930 INFO:teuthology.task.kernel:Updating grub on smithi120 to boot 6.4.0-rc5-g1c622d92b6f1 2023-06-23T00:52:19.930 DEBUG:teuthology.orchestra.run.smithi120:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-06-23T00:52:24.122 INFO:teuthology.orchestra.run.smithi120.stderr:Generating grub configuration file ... 2023-06-23T00:52:26.306 INFO:teuthology.orchestra.run.smithi120.stderr:done 2023-06-23T00:52:26.308 DEBUG:teuthology.orchestra.run.smithi120:> mktemp 2023-06-23T00:52:26.334 INFO:teuthology.orchestra.run.smithi120.stdout:/tmp/tmp.qpVa581iyy 2023-06-23T00:52:26.335 DEBUG:teuthology.orchestra.run.smithi120:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.qpVa581iyy 2023-06-23T00:52:26.418 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 0666 /tmp/tmp.qpVa581iyy 2023-06-23T00:52:26.588 DEBUG:teuthology.orchestra.remote:smithi120:/tmp/tmp.qpVa581iyy is 6KB 2023-06-23T00:52:26.638 DEBUG:teuthology.orchestra.run.smithi120:> rm -fr /tmp/tmp.qpVa581iyy 2023-06-23T00:52:26.652 DEBUG:teuthology.orchestra.run.smithi120:> sudo /bin/ls /boot/loader/entries || true 2023-06-23T00:52:26.721 INFO:teuthology.orchestra.run.smithi120.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-06-23T00:52:26.721 INFO:teuthology.orchestra.run.smithi120.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-06-23T00:52:26.722 INFO:teuthology.orchestra.run.smithi120.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-06-23T00:52:26.722 INFO:teuthology.orchestra.run.smithi120.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-06-23T00:52:26.722 INFO:teuthology.orchestra.run.smithi120.stdout:298e228f3b5e401e8ac0f5ae159a2e06-6.4.0-rc5-g1c622d92b6f1.conf 2023-06-23T00:52:26.723 DEBUG:teuthology.orchestra.run.smithi120:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-6.4.0-rc5-g1c622d92b6f1 2023-06-23T00:52:26.865 DEBUG:teuthology.orchestra.run.smithi120:> sudo shutdown -r now 2023-06-23T00:52:56.868 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-06-23T00:52:56.869 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2023-06-23T00:52:56.869 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T00:53:15.254 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.120 2023-06-23T00:53:45.263 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2023-06-23T00:53:45.264 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T00:54:03.633 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.120 2023-06-23T00:54:33.634 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2023-06-23T00:54:33.635 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T00:54:34.075 DEBUG:teuthology.orchestra.run.smithi120:> true 2023-06-23T00:54:34.444 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi120.front.sepia.ceph.com' 2023-06-23T00:54:34.444 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.4.0-rc5-g1c622d92b6f1"... 2023-06-23T00:54:34.445 DEBUG:teuthology.orchestra.run.smithi120:> uname -r 2023-06-23T00:54:34.479 INFO:teuthology.orchestra.run.smithi120.stdout:6.4.0-rc5-g1c622d92b6f1 2023-06-23T00:54:34.480 DEBUG:teuthology.task.kernel:current kernel version is 6.4.0-rc5-g1c622d92b6f1 vs 6.4.0-rc5-g1c622d92b6f1 2023-06-23T00:54:34.480 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-06-23T00:54:34.480 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-06-23T00:54:35.482 DEBUG:teuthology.parallel:result is None 2023-06-23T00:54:35.483 INFO:teuthology.run_tasks:Running task internal.base... 2023-06-23T00:54:35.491 INFO:teuthology.task.internal:Creating test directory... 2023-06-23T00:54:35.492 DEBUG:teuthology.orchestra.run.smithi120:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-06-23T00:54:35.499 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-06-23T00:54:35.504 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-06-23T00:54:35.533 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-06-23T00:54:35.543 INFO:teuthology.run_tasks:Running task internal.archive... 2023-06-23T00:54:35.577 INFO:teuthology.task.internal:Creating archive directory... 2023-06-23T00:54:35.578 DEBUG:teuthology.orchestra.run.smithi120:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-06-23T00:54:35.584 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-06-23T00:54:35.586 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-06-23T00:54:35.802 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-06-23T00:54:35.811 INFO:teuthology.task.internal:Enabling coredump saving... 2023-06-23T00:54:35.812 DEBUG:teuthology.orchestra.run.smithi134:> test -f /run/.containerenv -o -f /.dockerenv 2023-06-23T00:54:35.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T00:54:35.829 DEBUG:teuthology.orchestra.run.smithi157:> test -f /run/.containerenv -o -f /.dockerenv 2023-06-23T00:54:35.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T00:54:35.846 DEBUG:teuthology.orchestra.run.smithi120:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-06-23T00:54:35.851 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-06-23T00:54:35.872 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-06-23T00:54:35.915 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-23T00:54:35.929 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-23T00:54:35.933 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-23T00:54:35.951 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-23T00:54:36.009 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-23T00:54:36.048 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-23T00:54:36.053 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-06-23T00:54:36.063 INFO:teuthology.task.internal:Configuring sudo... 2023-06-23T00:54:36.063 DEBUG:teuthology.orchestra.run.smithi120:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-06-23T00:54:36.068 DEBUG:teuthology.orchestra.run.smithi134:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-06-23T00:54:36.071 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-06-23T00:54:36.126 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-06-23T00:54:36.135 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-06-23T00:54:36.136 DEBUG:teuthology.orchestra.run.smithi120:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-06-23T00:54:36.140 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-06-23T00:54:36.143 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-06-23T00:54:36.172 DEBUG:teuthology.orchestra.run.smithi120:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-23T00:54:36.284 DEBUG:teuthology.orchestra.run.smithi120:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-23T00:54:36.379 DEBUG:teuthology.orchestra.run.smithi120:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-23T00:54:36.449 DEBUG:teuthology.orchestra.run.smithi120:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-23T00:54:36.517 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2023-06-23T00:54:36.517 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-06-23T00:54:36.577 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-23T00:54:36.623 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-23T00:54:36.666 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-23T00:54:36.752 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-23T00:54:36.791 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-06-23T00:54:36.792 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-06-23T00:54:36.860 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-23T00:54:36.906 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-23T00:54:36.949 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-23T00:54:37.038 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-23T00:54:37.074 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-06-23T00:54:37.074 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-06-23T00:54:37.140 DEBUG:teuthology.orchestra.run.smithi120:> sudo service rsyslog restart 2023-06-23T00:54:37.145 DEBUG:teuthology.orchestra.run.smithi134:> sudo service rsyslog restart 2023-06-23T00:54:37.148 DEBUG:teuthology.orchestra.run.smithi157:> sudo service rsyslog restart 2023-06-23T00:54:37.211 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-23T00:54:37.232 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-23T00:54:37.244 INFO:teuthology.orchestra.run.smithi120.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-23T00:54:37.639 INFO:teuthology.run_tasks:Running task internal.timer... 2023-06-23T00:54:37.647 INFO:teuthology.task.internal:Starting timer... 2023-06-23T00:54:37.648 INFO:teuthology.run_tasks:Running task pcp... 2023-06-23T00:54:37.697 INFO:teuthology.run_tasks:Running task selinux... 2023-06-23T00:54:37.748 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-06-23T00:54:37.749 DEBUG:teuthology.orchestra.run.smithi120:> sudo service auditd rotate 2023-06-23T00:54:37.862 INFO:teuthology.orchestra.run.smithi120.stdout:Rotating logs: [ OK ] 2023-06-23T00:54:37.866 DEBUG:teuthology.orchestra.run.smithi134:> sudo service auditd rotate 2023-06-23T00:54:37.926 INFO:teuthology.orchestra.run.smithi134.stdout:Rotating logs: [ OK ] 2023-06-23T00:54:37.927 DEBUG:teuthology.orchestra.run.smithi157:> sudo service auditd rotate 2023-06-23T00:54:37.980 INFO:teuthology.orchestra.run.smithi157.stdout:Rotating logs: [ OK ] 2023-06-23T00:54:37.982 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-06-23T00:54:37.983 DEBUG:teuthology.orchestra.run.smithi120:> /usr/sbin/getenforce 2023-06-23T00:54:38.030 INFO:teuthology.orchestra.run.smithi120.stdout:Permissive 2023-06-23T00:54:38.031 DEBUG:teuthology.orchestra.run.smithi134:> /usr/sbin/getenforce 2023-06-23T00:54:38.056 INFO:teuthology.orchestra.run.smithi134.stdout:Permissive 2023-06-23T00:54:38.056 DEBUG:teuthology.orchestra.run.smithi157:> /usr/sbin/getenforce 2023-06-23T00:54:38.073 INFO:teuthology.orchestra.run.smithi157.stdout:Permissive 2023-06-23T00:54:38.073 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi120.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi134.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi157.front.sepia.ceph.com': 'permissive'} 2023-06-23T00:54:38.074 DEBUG:teuthology.orchestra.run.smithi120:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-23T00:54:38.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T00:54:38.137 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-23T00:54:38.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T00:54:38.162 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-23T00:54:38.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T00:54:38.189 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-06-23T00:54:38.189 DEBUG:teuthology.orchestra.run.smithi120:> sudo /usr/sbin/setenforce permissive 2023-06-23T00:54:38.253 DEBUG:teuthology.orchestra.run.smithi134:> sudo /usr/sbin/setenforce permissive 2023-06-23T00:54:38.277 DEBUG:teuthology.orchestra.run.smithi157:> sudo /usr/sbin/setenforce permissive 2023-06-23T00:54:38.312 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-06-23T00:54:38.325 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-06-23T00:54:38.331 INFO:teuthology.repo_utils:Fetching main from origin 2023-06-23T00:54:38.476 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-06-23T00:54:38.505 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-06-23T00:54:38.506 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi120.front.sepia.ceph.com,smithi134.front.sepia.ceph.com,smithi157.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-06-23T01:07:02.496 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi120.front.sepia.ceph.com'), Remote(name='ubuntu@smithi134.front.sepia.ceph.com'), Remote(name='ubuntu@smithi157.front.sepia.ceph.com')] 2023-06-23T01:07:02.498 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2023-06-23T01:07:02.499 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T01:07:02.581 DEBUG:teuthology.orchestra.run.smithi120:> true 2023-06-23T01:07:02.683 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2023-06-23T01:07:02.683 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T01:07:02.750 DEBUG:teuthology.orchestra.run.smithi134:> true 2023-06-23T01:07:02.816 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-06-23T01:07:02.817 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T01:07:02.888 DEBUG:teuthology.orchestra.run.smithi157:> true 2023-06-23T01:07:02.958 INFO:teuthology.run_tasks:Running task clock... 2023-06-23T01:07:02.979 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-06-23T01:07:02.980 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-06-23T01:07:02.981 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-23T01:07:02.985 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-06-23T01:07:02.985 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-23T01:07:02.988 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-06-23T01:07:02.988 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-23T01:07:03.018 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-06-23T01:07:03.030 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-06-23T01:07:03.033 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-06-23T01:07:03.045 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-06-23T01:07:03.047 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-06-23T01:07:03.078 INFO:teuthology.orchestra.run.smithi134.stderr:sudo: ntpd: command not found 2023-06-23T01:07:03.081 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-06-23T01:07:03.086 INFO:teuthology.orchestra.run.smithi157.stderr:sudo: ntpd: command not found 2023-06-23T01:07:03.089 INFO:teuthology.orchestra.run.smithi134.stdout:506 Cannot talk to daemon 2023-06-23T01:07:03.096 INFO:teuthology.orchestra.run.smithi157.stdout:506 Cannot talk to daemon 2023-06-23T01:07:03.103 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-06-23T01:07:03.110 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-06-23T01:07:03.117 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-06-23T01:07:03.124 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-06-23T01:07:03.140 INFO:teuthology.orchestra.run.smithi120.stderr:sudo: ntpd: command not found 2023-06-23T01:07:03.166 INFO:teuthology.orchestra.run.smithi120.stdout:506 Cannot talk to daemon 2023-06-23T01:07:03.186 INFO:teuthology.orchestra.run.smithi134.stderr:bash: ntpq: command not found 2023-06-23T01:07:03.189 INFO:teuthology.orchestra.run.smithi134.stdout:210 Number of sources = 4 2023-06-23T01:07:03.190 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-23T01:07:03.190 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2023-06-23T01:07:03.190 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T01:07:03.190 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T01:07:03.190 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T01:07:03.190 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T01:07:03.195 INFO:teuthology.orchestra.run.smithi157.stderr:bash: ntpq: command not found 2023-06-23T01:07:03.197 INFO:teuthology.orchestra.run.smithi157.stdout:210 Number of sources = 4 2023-06-23T01:07:03.198 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-23T01:07:03.198 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2023-06-23T01:07:03.198 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T01:07:03.198 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T01:07:03.198 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T01:07:03.199 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T01:07:03.199 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-06-23T01:07:03.232 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-06-23T01:07:03.317 INFO:teuthology.orchestra.run.smithi120.stderr:bash: ntpq: command not found 2023-06-23T01:07:03.321 INFO:teuthology.orchestra.run.smithi120.stdout:210 Number of sources = 4 2023-06-23T01:07:03.322 INFO:teuthology.orchestra.run.smithi120.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-23T01:07:03.322 INFO:teuthology.orchestra.run.smithi120.stdout:=============================================================================== 2023-06-23T01:07:03.322 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T01:07:03.323 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T01:07:03.323 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T01:07:03.323 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T01:07:03.325 INFO:teuthology.run_tasks:Running task pexec... 2023-06-23T01:07:03.338 INFO:teuthology.task.pexec:Executing custom commands... 2023-06-23T01:07:03.338 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi120.front.sepia.ceph.com 2023-06-23T01:07:03.339 DEBUG:teuthology.orchestra.run.smithi120:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-06-23T01:07:03.339 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi134.front.sepia.ceph.com 2023-06-23T01:07:03.339 DEBUG:teuthology.orchestra.run.smithi134:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-06-23T01:07:03.340 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi157.front.sepia.ceph.com 2023-06-23T01:07:03.340 DEBUG:teuthology.orchestra.run.smithi157:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-06-23T01:07:03.726 INFO:teuthology.orchestra.run.smithi134.stdout:Updating Subscription Management repositories. 2023-06-23T01:07:03.730 INFO:teuthology.orchestra.run.smithi157.stdout:Updating Subscription Management repositories. 2023-06-23T01:07:03.838 INFO:teuthology.orchestra.run.smithi120.stdout:Updating Subscription Management repositories. 2023-06-23T01:07:04.511 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:05:48 ago on Fri 23 Jun 2023 01:01:16 AM UTC. 2023-06-23T01:07:04.524 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:27 ago on Fri 23 Jun 2023 01:05:37 AM UTC. 2023-06-23T01:07:04.824 INFO:teuthology.orchestra.run.smithi120.stdout:Last metadata expiration check: 0:03:37 ago on Fri 23 Jun 2023 01:03:27 AM UTC. 2023-06-23T01:07:06.504 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2023-06-23T01:07:06.505 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2023-06-23T01:07:06.505 INFO:teuthology.orchestra.run.smithi134.stdout: Package Architecture Version Repository Size 2023-06-23T01:07:06.505 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2023-06-23T01:07:06.505 INFO:teuthology.orchestra.run.smithi134.stdout:Resetting modules: 2023-06-23T01:07:06.506 INFO:teuthology.orchestra.run.smithi134.stdout: container-tools 2023-06-23T01:07:06.506 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-06-23T01:07:06.506 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2023-06-23T01:07:06.506 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2023-06-23T01:07:06.506 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-06-23T01:07:06.542 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-06-23T01:07:06.543 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-06-23T01:07:06.543 INFO:teuthology.orchestra.run.smithi157.stdout: Package Architecture Version Repository Size 2023-06-23T01:07:06.544 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-06-23T01:07:06.544 INFO:teuthology.orchestra.run.smithi157.stdout:Resetting modules: 2023-06-23T01:07:06.544 INFO:teuthology.orchestra.run.smithi157.stdout: container-tools 2023-06-23T01:07:06.544 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-06-23T01:07:06.544 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2023-06-23T01:07:06.544 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-06-23T01:07:06.544 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-06-23T01:07:06.925 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2023-06-23T01:07:06.926 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2023-06-23T01:07:06.926 INFO:teuthology.orchestra.run.smithi120.stdout: Package Architecture Version Repository Size 2023-06-23T01:07:06.927 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2023-06-23T01:07:06.927 INFO:teuthology.orchestra.run.smithi120.stdout:Resetting modules: 2023-06-23T01:07:06.927 INFO:teuthology.orchestra.run.smithi120.stdout: container-tools 2023-06-23T01:07:06.927 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:07:06.927 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction Summary 2023-06-23T01:07:06.927 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2023-06-23T01:07:06.927 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:07:07.367 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:05:51 ago on Fri 23 Jun 2023 01:01:16 AM UTC. 2023-06-23T01:07:07.387 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:30 ago on Fri 23 Jun 2023 01:05:37 AM UTC. 2023-06-23T01:07:08.071 INFO:teuthology.orchestra.run.smithi120.stdout:Last metadata expiration check: 0:03:40 ago on Fri 23 Jun 2023 01:03:27 AM UTC. 2023-06-23T01:07:09.312 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2023-06-23T01:07:09.344 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-06-23T01:07:09.768 INFO:teuthology.orchestra.run.smithi134.stdout:Updating Subscription Management repositories. 2023-06-23T01:07:09.824 INFO:teuthology.orchestra.run.smithi157.stdout:Updating Subscription Management repositories. 2023-06-23T01:07:09.860 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2023-06-23T01:07:10.417 INFO:teuthology.orchestra.run.smithi120.stdout:Updating Subscription Management repositories. 2023-06-23T01:07:10.511 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:05:54 ago on Fri 23 Jun 2023 01:01:16 AM UTC. 2023-06-23T01:07:10.549 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:33 ago on Fri 23 Jun 2023 01:05:37 AM UTC. 2023-06-23T01:07:11.376 INFO:teuthology.orchestra.run.smithi120.stdout:Last metadata expiration check: 0:03:43 ago on Fri 23 Jun 2023 01:03:27 AM UTC. 2023-06-23T01:07:12.497 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2023-06-23T01:07:12.527 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-06-23T01:07:12.545 INFO:teuthology.orchestra.run.smithi134.stdout:========================================================================================================= 2023-06-23T01:07:12.546 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repository Size 2023-06-23T01:07:12.546 INFO:teuthology.orchestra.run.smithi134.stdout:========================================================================================================= 2023-06-23T01:07:12.546 INFO:teuthology.orchestra.run.smithi134.stdout:Installing group/module packages: 2023-06-23T01:07:12.546 INFO:teuthology.orchestra.run.smithi134.stdout: buildah x86_64 1.21.4-2.module+el8.4.0+12050+ef972f71 rhel-8-for-x86_64-appstream-rpms 8.2 M 2023-06-23T01:07:12.547 INFO:teuthology.orchestra.run.smithi134.stdout: cockpit-podman noarch 32-2.module+el8.4.0+11990+22932769 rhel-8-for-x86_64-appstream-rpms 410 k 2023-06-23T01:07:12.547 INFO:teuthology.orchestra.run.smithi134.stdout: crun x86_64 0.20.1-1.module+el8.4.0+11822+6cc1e7d7 rhel-8-for-x86_64-appstream-rpms 192 k 2023-06-23T01:07:12.547 INFO:teuthology.orchestra.run.smithi134.stdout: skopeo x86_64 1:1.3.1-5.module+el8.4.0+11990+22932769 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-06-23T01:07:12.548 INFO:teuthology.orchestra.run.smithi134.stdout: toolbox noarch 0.0.8-1.module+el8.4.0+11822+6cc1e7d7 rhel-8-for-x86_64-appstream-rpms 16 k 2023-06-23T01:07:12.548 INFO:teuthology.orchestra.run.smithi134.stdout: udica noarch 0.2.4-2.module+el8.4.0+11822+6cc1e7d7 rhel-8-for-x86_64-appstream-rpms 51 k 2023-06-23T01:07:12.548 INFO:teuthology.orchestra.run.smithi134.stdout:Installing dependencies: 2023-06-23T01:07:12.548 INFO:teuthology.orchestra.run.smithi134.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-06-23T01:07:12.548 INFO:teuthology.orchestra.run.smithi134.stdout:Installing module profiles: 2023-06-23T01:07:12.548 INFO:teuthology.orchestra.run.smithi134.stdout: container-tools/common 2023-06-23T01:07:12.548 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-06-23T01:07:12.549 INFO:teuthology.orchestra.run.smithi134.stdout:Enabling module streams: 2023-06-23T01:07:12.549 INFO:teuthology.orchestra.run.smithi134.stdout: container-tools rhel8 2023-06-23T01:07:12.549 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-06-23T01:07:12.549 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2023-06-23T01:07:12.549 INFO:teuthology.orchestra.run.smithi134.stdout:========================================================================================================= 2023-06-23T01:07:12.549 INFO:teuthology.orchestra.run.smithi134.stdout:Install 7 Packages 2023-06-23T01:07:12.549 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-06-23T01:07:12.550 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 16 M 2023-06-23T01:07:12.550 INFO:teuthology.orchestra.run.smithi134.stdout:Installed size: 57 M 2023-06-23T01:07:12.550 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2023-06-23T01:07:12.577 INFO:teuthology.orchestra.run.smithi157.stdout:========================================================================================================= 2023-06-23T01:07:12.577 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2023-06-23T01:07:12.577 INFO:teuthology.orchestra.run.smithi157.stdout:========================================================================================================= 2023-06-23T01:07:12.577 INFO:teuthology.orchestra.run.smithi157.stdout:Installing group/module packages: 2023-06-23T01:07:12.577 INFO:teuthology.orchestra.run.smithi157.stdout: buildah x86_64 1.21.4-2.module+el8.4.0+12050+ef972f71 rhel-8-for-x86_64-appstream-rpms 8.2 M 2023-06-23T01:07:12.578 INFO:teuthology.orchestra.run.smithi157.stdout: cockpit-podman noarch 32-2.module+el8.4.0+11990+22932769 rhel-8-for-x86_64-appstream-rpms 410 k 2023-06-23T01:07:12.578 INFO:teuthology.orchestra.run.smithi157.stdout: crun x86_64 0.20.1-1.module+el8.4.0+11822+6cc1e7d7 rhel-8-for-x86_64-appstream-rpms 192 k 2023-06-23T01:07:12.578 INFO:teuthology.orchestra.run.smithi157.stdout: skopeo x86_64 1:1.3.1-5.module+el8.4.0+11990+22932769 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-06-23T01:07:12.578 INFO:teuthology.orchestra.run.smithi157.stdout: toolbox noarch 0.0.8-1.module+el8.4.0+11822+6cc1e7d7 rhel-8-for-x86_64-appstream-rpms 16 k 2023-06-23T01:07:12.578 INFO:teuthology.orchestra.run.smithi157.stdout: udica noarch 0.2.4-2.module+el8.4.0+11822+6cc1e7d7 rhel-8-for-x86_64-appstream-rpms 51 k 2023-06-23T01:07:12.578 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2023-06-23T01:07:12.578 INFO:teuthology.orchestra.run.smithi157.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-06-23T01:07:12.579 INFO:teuthology.orchestra.run.smithi157.stdout:Installing module profiles: 2023-06-23T01:07:12.579 INFO:teuthology.orchestra.run.smithi157.stdout: container-tools/common 2023-06-23T01:07:12.579 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-06-23T01:07:12.579 INFO:teuthology.orchestra.run.smithi157.stdout:Enabling module streams: 2023-06-23T01:07:12.579 INFO:teuthology.orchestra.run.smithi157.stdout: container-tools rhel8 2023-06-23T01:07:12.579 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-06-23T01:07:12.579 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2023-06-23T01:07:12.579 INFO:teuthology.orchestra.run.smithi157.stdout:========================================================================================================= 2023-06-23T01:07:12.580 INFO:teuthology.orchestra.run.smithi157.stdout:Install 7 Packages 2023-06-23T01:07:12.580 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-06-23T01:07:12.580 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 16 M 2023-06-23T01:07:12.580 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 57 M 2023-06-23T01:07:12.581 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2023-06-23T01:07:12.600 INFO:teuthology.orchestra.run.smithi134.stdout:(1/7): cockpit-podman-32-2.module+el8.4.0+11990 7.6 MB/s | 410 kB 00:00 2023-06-23T01:07:12.604 INFO:teuthology.orchestra.run.smithi134.stdout:(2/7): yajl-2.1.0-10.el8.x86_64.rpm 715 kB/s | 41 kB 00:00 2023-06-23T01:07:12.617 INFO:teuthology.orchestra.run.smithi157.stdout:(1/7): cockpit-podman-32-2.module+el8.4.0+11990 11 MB/s | 410 kB 00:00 2023-06-23T01:07:12.620 INFO:teuthology.orchestra.run.smithi134.stdout:(3/7): crun-0.20.1-1.module+el8.4.0+11822+6cc1e 2.6 MB/s | 192 kB 00:00 2023-06-23T01:07:12.622 INFO:teuthology.orchestra.run.smithi157.stdout:(2/7): yajl-2.1.0-10.el8.x86_64.rpm 942 kB/s | 41 kB 00:00 2023-06-23T01:07:12.636 INFO:teuthology.orchestra.run.smithi134.stdout:(4/7): toolbox-0.0.8-1.module+el8.4.0+11822+6cc 444 kB/s | 16 kB 00:00 2023-06-23T01:07:12.648 INFO:teuthology.orchestra.run.smithi157.stdout:(3/7): crun-0.20.1-1.module+el8.4.0+11822+6cc1e 2.7 MB/s | 192 kB 00:00 2023-06-23T01:07:12.653 INFO:teuthology.orchestra.run.smithi157.stdout:(4/7): toolbox-0.0.8-1.module+el8.4.0+11822+6cc 442 kB/s | 16 kB 00:00 2023-06-23T01:07:12.682 INFO:teuthology.orchestra.run.smithi134.stdout:(5/7): udica-0.2.4-2.module+el8.4.0+11822+6cc1e 1.1 MB/s | 51 kB 00:00 2023-06-23T01:07:12.690 INFO:teuthology.orchestra.run.smithi157.stdout:(5/7): udica-0.2.4-2.module+el8.4.0+11822+6cc1e 1.3 MB/s | 51 kB 00:00 2023-06-23T01:07:12.741 INFO:teuthology.orchestra.run.smithi157.stdout:(6/7): skopeo-1.3.1-5.module+el8.4.0+11990+2293 59 MB/s | 7.0 MB 00:00 2023-06-23T01:07:12.745 INFO:teuthology.orchestra.run.smithi134.stdout:(6/7): skopeo-1.3.1-5.module+el8.4.0+11990+2293 50 MB/s | 7.0 MB 00:00 2023-06-23T01:07:12.767 INFO:teuthology.orchestra.run.smithi134.stdout:(7/7): buildah-1.21.4-2.module+el8.4.0+12050+ef 55 MB/s | 8.2 MB 00:00 2023-06-23T01:07:12.768 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2023-06-23T01:07:12.768 INFO:teuthology.orchestra.run.smithi134.stdout:Total 72 MB/s | 16 MB 00:00 2023-06-23T01:07:12.774 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2023-06-23T01:07:12.780 INFO:teuthology.orchestra.run.smithi157.stdout:(7/7): buildah-1.21.4-2.module+el8.4.0+12050+ef 62 MB/s | 8.2 MB 00:00 2023-06-23T01:07:12.781 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2023-06-23T01:07:12.781 INFO:teuthology.orchestra.run.smithi157.stdout:Total 78 MB/s | 16 MB 00:00 2023-06-23T01:07:12.786 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2023-06-23T01:07:12.790 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2023-06-23T01:07:12.791 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2023-06-23T01:07:12.803 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2023-06-23T01:07:12.804 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2023-06-23T01:07:13.320 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2023-06-23T01:07:13.324 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2023-06-23T01:07:13.345 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2023-06-23T01:07:13.350 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2023-06-23T01:07:13.486 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2023-06-23T01:07:13.537 INFO:teuthology.orchestra.run.smithi120.stdout:========================================================================================================= 2023-06-23T01:07:13.538 INFO:teuthology.orchestra.run.smithi120.stdout: Package Arch Version Repository Size 2023-06-23T01:07:13.538 INFO:teuthology.orchestra.run.smithi120.stdout:========================================================================================================= 2023-06-23T01:07:13.538 INFO:teuthology.orchestra.run.smithi120.stdout:Installing group/module packages: 2023-06-23T01:07:13.538 INFO:teuthology.orchestra.run.smithi120.stdout: buildah x86_64 1.21.4-2.module+el8.4.0+12050+ef972f71 rhel-8-for-x86_64-appstream-rpms 8.2 M 2023-06-23T01:07:13.538 INFO:teuthology.orchestra.run.smithi120.stdout: cockpit-podman noarch 32-2.module+el8.4.0+11990+22932769 rhel-8-for-x86_64-appstream-rpms 410 k 2023-06-23T01:07:13.539 INFO:teuthology.orchestra.run.smithi120.stdout: crun x86_64 0.20.1-1.module+el8.4.0+11822+6cc1e7d7 rhel-8-for-x86_64-appstream-rpms 192 k 2023-06-23T01:07:13.539 INFO:teuthology.orchestra.run.smithi120.stdout: skopeo x86_64 1:1.3.1-5.module+el8.4.0+11990+22932769 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-06-23T01:07:13.539 INFO:teuthology.orchestra.run.smithi120.stdout: toolbox noarch 0.0.8-1.module+el8.4.0+11822+6cc1e7d7 rhel-8-for-x86_64-appstream-rpms 16 k 2023-06-23T01:07:13.539 INFO:teuthology.orchestra.run.smithi120.stdout: udica noarch 0.2.4-2.module+el8.4.0+11822+6cc1e7d7 rhel-8-for-x86_64-appstream-rpms 51 k 2023-06-23T01:07:13.539 INFO:teuthology.orchestra.run.smithi120.stdout:Installing dependencies: 2023-06-23T01:07:13.539 INFO:teuthology.orchestra.run.smithi120.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-06-23T01:07:13.539 INFO:teuthology.orchestra.run.smithi120.stdout:Installing module profiles: 2023-06-23T01:07:13.539 INFO:teuthology.orchestra.run.smithi120.stdout: container-tools/common 2023-06-23T01:07:13.540 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:07:13.540 INFO:teuthology.orchestra.run.smithi120.stdout:Enabling module streams: 2023-06-23T01:07:13.540 INFO:teuthology.orchestra.run.smithi120.stdout: container-tools rhel8 2023-06-23T01:07:13.540 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:07:13.540 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction Summary 2023-06-23T01:07:13.541 INFO:teuthology.orchestra.run.smithi120.stdout:========================================================================================================= 2023-06-23T01:07:13.541 INFO:teuthology.orchestra.run.smithi120.stdout:Install 7 Packages 2023-06-23T01:07:13.541 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:07:13.542 INFO:teuthology.orchestra.run.smithi120.stdout:Total download size: 16 M 2023-06-23T01:07:13.542 INFO:teuthology.orchestra.run.smithi120.stdout:Installed size: 57 M 2023-06-23T01:07:13.542 INFO:teuthology.orchestra.run.smithi120.stdout:Downloading Packages: 2023-06-23T01:07:13.572 INFO:teuthology.orchestra.run.smithi120.stdout:(1/7): crun-0.20.1-1.module+el8.4.0+11822+6cc1e 5.9 MB/s | 192 kB 00:00 2023-06-23T01:07:13.581 INFO:teuthology.orchestra.run.smithi120.stdout:(2/7): cockpit-podman-32-2.module+el8.4.0+11990 9.9 MB/s | 410 kB 00:00 2023-06-23T01:07:13.583 INFO:teuthology.orchestra.run.smithi120.stdout:(3/7): yajl-2.1.0-10.el8.x86_64.rpm 955 kB/s | 41 kB 00:00 2023-06-23T01:07:13.586 INFO:teuthology.orchestra.run.smithi120.stdout:(4/7): toolbox-0.0.8-1.module+el8.4.0+11822+6cc 1.1 MB/s | 16 kB 00:00 2023-06-23T01:07:13.601 INFO:teuthology.orchestra.run.smithi120.stdout:(5/7): udica-0.2.4-2.module+el8.4.0+11822+6cc1e 3.3 MB/s | 51 kB 00:00 2023-06-23T01:07:13.726 INFO:teuthology.orchestra.run.smithi120.stdout:(6/7): skopeo-1.3.1-5.module+el8.4.0+11990+2293 49 MB/s | 7.0 MB 00:00 2023-06-23T01:07:13.761 INFO:teuthology.orchestra.run.smithi120.stdout:(7/7): buildah-1.21.4-2.module+el8.4.0+12050+ef 46 MB/s | 8.2 MB 00:00 2023-06-23T01:07:13.761 INFO:teuthology.orchestra.run.smithi120.stdout:-------------------------------------------------------------------------------- 2023-06-23T01:07:13.762 INFO:teuthology.orchestra.run.smithi120.stdout:Total 71 MB/s | 16 MB 00:00 2023-06-23T01:07:13.771 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction check 2023-06-23T01:07:13.790 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction check succeeded. 2023-06-23T01:07:13.791 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction test 2023-06-23T01:07:13.800 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2023-06-23T01:07:13.810 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2023-06-23T01:07:14.025 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : yajl-2.1.0-10.el8.x86_64 1/7 2023-06-23T01:07:14.038 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : yajl-2.1.0-10.el8.x86_64 1/7 2023-06-23T01:07:14.216 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : crun-0.20.1-1.module+el8.4.0+11822+6cc1e7d7.x86_64 2/7 2023-06-23T01:07:14.231 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : crun-0.20.1-1.module+el8.4.0+11822+6cc1e7d7.x86_64 2/7 2023-06-23T01:07:14.323 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction test succeeded. 2023-06-23T01:07:14.328 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction 2023-06-23T01:07:14.757 INFO:teuthology.orchestra.run.smithi120.stdout: Preparing : 1/1 2023-06-23T01:07:14.986 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : yajl-2.1.0-10.el8.x86_64 1/7 2023-06-23T01:07:15.090 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : udica-0.2.4-2.module+el8.4.0+11822+6cc1e7d7.noarch 3/7 2023-06-23T01:07:15.158 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : udica-0.2.4-2.module+el8.4.0+11822+6cc1e7d7.noarch 3/7 2023-06-23T01:07:15.180 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : crun-0.20.1-1.module+el8.4.0+11822+6cc1e7d7.x86_64 2/7 2023-06-23T01:07:15.871 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : buildah-1.21.4-2.module+el8.4.0+12050+ef972f71.x86_6 4/7 2023-06-23T01:07:15.971 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : buildah-1.21.4-2.module+el8.4.0+12050+ef972f71.x86_6 4/7 2023-06-23T01:07:16.026 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : skopeo-1:1.3.1-5.module+el8.4.0+11990+22932769.x86_6 5/7 2023-06-23T01:07:16.176 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : toolbox-0.0.8-1.module+el8.4.0+11822+6cc1e7d7.noarch 6/7 2023-06-23T01:07:16.178 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : udica-0.2.4-2.module+el8.4.0+11822+6cc1e7d7.noarch 3/7 2023-06-23T01:07:16.181 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : skopeo-1:1.3.1-5.module+el8.4.0+11990+22932769.x86_6 5/7 2023-06-23T01:07:16.318 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : toolbox-0.0.8-1.module+el8.4.0+11822+6cc1e7d7.noarch 6/7 2023-06-23T01:07:16.346 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cockpit-podman-32-2.module+el8.4.0+11990+22932769.no 7/7 2023-06-23T01:07:16.485 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cockpit-podman-32-2.module+el8.4.0+11990+22932769.no 7/7 2023-06-23T01:07:16.615 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cockpit-podman-32-2.module+el8.4.0+11990+22932769.no 7/7 2023-06-23T01:07:16.615 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 1/7 2023-06-23T01:07:16.615 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : crun-0.20.1-1.module+el8.4.0+11822+6cc1e7d7.x86_64 2/7 2023-06-23T01:07:16.616 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cockpit-podman-32-2.module+el8.4.0+11990+22932769.no 3/7 2023-06-23T01:07:16.616 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : toolbox-0.0.8-1.module+el8.4.0+11822+6cc1e7d7.noarch 4/7 2023-06-23T01:07:16.616 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : skopeo-1:1.3.1-5.module+el8.4.0+11990+22932769.x86_6 5/7 2023-06-23T01:07:16.616 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : buildah-1.21.4-2.module+el8.4.0+12050+ef972f71.x86_6 6/7 2023-06-23T01:07:16.771 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cockpit-podman-32-2.module+el8.4.0+11990+22932769.no 7/7 2023-06-23T01:07:16.772 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 1/7 2023-06-23T01:07:16.772 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : crun-0.20.1-1.module+el8.4.0+11822+6cc1e7d7.x86_64 2/7 2023-06-23T01:07:16.772 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cockpit-podman-32-2.module+el8.4.0+11990+22932769.no 3/7 2023-06-23T01:07:16.773 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : toolbox-0.0.8-1.module+el8.4.0+11822+6cc1e7d7.noarch 4/7 2023-06-23T01:07:16.773 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : skopeo-1:1.3.1-5.module+el8.4.0+11990+22932769.x86_6 5/7 2023-06-23T01:07:16.773 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : buildah-1.21.4-2.module+el8.4.0+12050+ef972f71.x86_6 6/7 2023-06-23T01:07:17.042 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : buildah-1.21.4-2.module+el8.4.0+12050+ef972f71.x86_6 4/7 2023-06-23T01:07:17.219 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : skopeo-1:1.3.1-5.module+el8.4.0+11990+22932769.x86_6 5/7 2023-06-23T01:07:17.406 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : toolbox-0.0.8-1.module+el8.4.0+11822+6cc1e7d7.noarch 6/7 2023-06-23T01:07:17.548 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cockpit-podman-32-2.module+el8.4.0+11990+22932769.no 7/7 2023-06-23T01:07:17.662 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : udica-0.2.4-2.module+el8.4.0+11822+6cc1e7d7.noarch 7/7 2023-06-23T01:07:17.662 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:06:01 ago on Fri 23 Jun 2023 01:01:16 AM UTC. 2023-06-23T01:07:17.893 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : udica-0.2.4-2.module+el8.4.0+11822+6cc1e7d7.noarch 7/7 2023-06-23T01:07:17.894 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:40 ago on Fri 23 Jun 2023 01:05:37 AM UTC. 2023-06-23T01:07:17.915 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: cockpit-podman-32-2.module+el8.4.0+11990+22932769.no 7/7 2023-06-23T01:07:17.915 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 1/7 2023-06-23T01:07:17.915 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : crun-0.20.1-1.module+el8.4.0+11822+6cc1e7d7.x86_64 2/7 2023-06-23T01:07:17.915 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cockpit-podman-32-2.module+el8.4.0+11990+22932769.no 3/7 2023-06-23T01:07:17.916 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : toolbox-0.0.8-1.module+el8.4.0+11822+6cc1e7d7.noarch 4/7 2023-06-23T01:07:17.916 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : skopeo-1:1.3.1-5.module+el8.4.0+11990+22932769.x86_6 5/7 2023-06-23T01:07:17.916 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : buildah-1.21.4-2.module+el8.4.0+12050+ef972f71.x86_6 6/7 2023-06-23T01:07:19.230 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : udica-0.2.4-2.module+el8.4.0+11822+6cc1e7d7.noarch 7/7 2023-06-23T01:07:19.230 INFO:teuthology.orchestra.run.smithi120.stdout:Last metadata expiration check: 0:03:51 ago on Fri 23 Jun 2023 01:03:27 AM UTC. 2023-06-23T01:07:19.944 INFO:teuthology.orchestra.run.smithi134.stdout:Installed products updated. 2023-06-23T01:07:20.058 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-06-23T01:07:20.059 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2023-06-23T01:07:20.059 INFO:teuthology.orchestra.run.smithi134.stdout: buildah-1.21.4-2.module+el8.4.0+12050+ef972f71.x86_64 2023-06-23T01:07:20.059 INFO:teuthology.orchestra.run.smithi134.stdout: cockpit-podman-32-2.module+el8.4.0+11990+22932769.noarch 2023-06-23T01:07:20.059 INFO:teuthology.orchestra.run.smithi134.stdout: crun-0.20.1-1.module+el8.4.0+11822+6cc1e7d7.x86_64 2023-06-23T01:07:20.059 INFO:teuthology.orchestra.run.smithi134.stdout: skopeo-1:1.3.1-5.module+el8.4.0+11990+22932769.x86_64 2023-06-23T01:07:20.059 INFO:teuthology.orchestra.run.smithi134.stdout: toolbox-0.0.8-1.module+el8.4.0+11822+6cc1e7d7.noarch 2023-06-23T01:07:20.060 INFO:teuthology.orchestra.run.smithi134.stdout: udica-0.2.4-2.module+el8.4.0+11822+6cc1e7d7.noarch 2023-06-23T01:07:20.060 INFO:teuthology.orchestra.run.smithi134.stdout: yajl-2.1.0-10.el8.x86_64 2023-06-23T01:07:20.060 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-06-23T01:07:20.060 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2023-06-23T01:07:20.193 DEBUG:teuthology.parallel:result is None 2023-06-23T01:07:20.216 INFO:teuthology.orchestra.run.smithi157.stdout:Installed products updated. 2023-06-23T01:07:20.323 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-06-23T01:07:20.323 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2023-06-23T01:07:20.323 INFO:teuthology.orchestra.run.smithi157.stdout: buildah-1.21.4-2.module+el8.4.0+12050+ef972f71.x86_64 2023-06-23T01:07:20.323 INFO:teuthology.orchestra.run.smithi157.stdout: cockpit-podman-32-2.module+el8.4.0+11990+22932769.noarch 2023-06-23T01:07:20.323 INFO:teuthology.orchestra.run.smithi157.stdout: crun-0.20.1-1.module+el8.4.0+11822+6cc1e7d7.x86_64 2023-06-23T01:07:20.324 INFO:teuthology.orchestra.run.smithi157.stdout: skopeo-1:1.3.1-5.module+el8.4.0+11990+22932769.x86_64 2023-06-23T01:07:20.324 INFO:teuthology.orchestra.run.smithi157.stdout: toolbox-0.0.8-1.module+el8.4.0+11822+6cc1e7d7.noarch 2023-06-23T01:07:20.324 INFO:teuthology.orchestra.run.smithi157.stdout: udica-0.2.4-2.module+el8.4.0+11822+6cc1e7d7.noarch 2023-06-23T01:07:20.324 INFO:teuthology.orchestra.run.smithi157.stdout: yajl-2.1.0-10.el8.x86_64 2023-06-23T01:07:20.324 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-06-23T01:07:20.324 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-06-23T01:07:20.468 DEBUG:teuthology.parallel:result is None 2023-06-23T01:07:21.471 INFO:teuthology.orchestra.run.smithi120.stdout:Installed products updated. 2023-06-23T01:07:21.591 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:07:21.591 INFO:teuthology.orchestra.run.smithi120.stdout:Installed: 2023-06-23T01:07:21.592 INFO:teuthology.orchestra.run.smithi120.stdout: buildah-1.21.4-2.module+el8.4.0+12050+ef972f71.x86_64 2023-06-23T01:07:21.596 INFO:teuthology.orchestra.run.smithi120.stdout: cockpit-podman-32-2.module+el8.4.0+11990+22932769.noarch 2023-06-23T01:07:21.596 INFO:teuthology.orchestra.run.smithi120.stdout: crun-0.20.1-1.module+el8.4.0+11822+6cc1e7d7.x86_64 2023-06-23T01:07:21.596 INFO:teuthology.orchestra.run.smithi120.stdout: skopeo-1:1.3.1-5.module+el8.4.0+11990+22932769.x86_64 2023-06-23T01:07:21.596 INFO:teuthology.orchestra.run.smithi120.stdout: toolbox-0.0.8-1.module+el8.4.0+11822+6cc1e7d7.noarch 2023-06-23T01:07:21.596 INFO:teuthology.orchestra.run.smithi120.stdout: udica-0.2.4-2.module+el8.4.0+11822+6cc1e7d7.noarch 2023-06-23T01:07:21.597 INFO:teuthology.orchestra.run.smithi120.stdout: yajl-2.1.0-10.el8.x86_64 2023-06-23T01:07:21.597 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:07:21.597 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2023-06-23T01:07:21.826 DEBUG:teuthology.parallel:result is None 2023-06-23T01:07:21.827 INFO:teuthology.run_tasks:Running task install... 2023-06-23T01:07:21.836 DEBUG:teuthology.task.install:project ceph 2023-06-23T01:07:21.838 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c'}} 2023-06-23T01:07:21.839 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c'} 2023-06-23T01:07:21.839 INFO:teuthology.task.install:Using flavor: default 2023-06-23T01:07:21.845 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-06-23T01:07:21.846 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-06-23T01:07:21.846 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c', 'tag': None, 'wait_for_package': False} 2023-06-23T01:07:21.847 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=42ddc46e6e6582d8e29a5a982a81886fe028be2c 2023-06-23T01:07:21.849 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c', 'tag': None, 'wait_for_package': False} 2023-06-23T01:07:21.849 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=42ddc46e6e6582d8e29a5a982a81886fe028be2c 2023-06-23T01:07:21.850 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c', 'tag': None, 'wait_for_package': False} 2023-06-23T01:07:21.851 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=42ddc46e6e6582d8e29a5a982a81886fe028be2c 2023-06-23T01:07:22.000 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/ 2023-06-23T01:07:22.000 INFO:teuthology.task.install.rpm:Package version is 17.2.6-719.g42ddc46e 2023-06-23T01:07:22.015 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/ 2023-06-23T01:07:22.015 INFO:teuthology.task.install.rpm:Package version is 17.2.6-719.g42ddc46e 2023-06-23T01:07:22.017 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/ 2023-06-23T01:07:22.017 INFO:teuthology.task.install.rpm:Package version is 17.2.6-719.g42ddc46e 2023-06-23T01:07:22.121 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-06-23T01:07:22.122 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-06-23T01:07:22.122 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-06-23T01:07:22.147 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-06-23T01:07:22.148 DEBUG:teuthology.orchestra.run.smithi134:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/42ddc46e6e6582d8e29a5a982a81886fe028be2c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-06-23T01:07:22.172 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-06-23T01:07:22.173 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2023-06-23T01:07:22.173 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-06-23T01:07:22.181 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-06-23T01:07:22.182 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-06-23T01:07:22.182 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-06-23T01:07:22.205 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-06-23T01:07:22.206 DEBUG:teuthology.orchestra.run.smithi157:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/42ddc46e6e6582d8e29a5a982a81886fe028be2c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-06-23T01:07:22.210 DEBUG:teuthology.orchestra.run.smithi134:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-06-23T01:07:22.233 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-06-23T01:07:22.234 DEBUG:teuthology.orchestra.run.smithi120:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/42ddc46e6e6582d8e29a5a982a81886fe028be2c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-06-23T01:07:22.271 DEBUG:teuthology.orchestra.run.smithi157:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-06-23T01:07:22.283 DEBUG:teuthology.orchestra.run.smithi134:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-06-23T01:07:22.334 DEBUG:teuthology.orchestra.run.smithi120:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-06-23T01:07:22.345 DEBUG:teuthology.orchestra.run.smithi157:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-06-23T01:07:22.346 INFO:teuthology.orchestra.run.smithi134.stdout:check_obsoletes = 1 2023-06-23T01:07:22.347 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum clean all 2023-06-23T01:07:22.407 INFO:teuthology.orchestra.run.smithi157.stdout:check_obsoletes = 1 2023-06-23T01:07:22.409 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum clean all 2023-06-23T01:07:22.421 DEBUG:teuthology.orchestra.run.smithi120:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-06-23T01:07:22.481 INFO:teuthology.orchestra.run.smithi120.stdout:check_obsoletes = 1 2023-06-23T01:07:22.484 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum clean all 2023-06-23T01:07:22.745 INFO:teuthology.orchestra.run.smithi134.stdout:Updating Subscription Management repositories. 2023-06-23T01:07:22.831 INFO:teuthology.orchestra.run.smithi157.stdout:Updating Subscription Management repositories. 2023-06-23T01:07:22.940 INFO:teuthology.orchestra.run.smithi120.stdout:Updating Subscription Management repositories. 2023-06-23T01:07:23.112 INFO:teuthology.orchestra.run.smithi134.stdout:44 files removed 2023-06-23T01:07:23.164 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-06-23T01:07:23.202 INFO:teuthology.orchestra.run.smithi157.stdout:44 files removed 2023-06-23T01:07:23.260 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-06-23T01:07:23.342 INFO:teuthology.orchestra.run.smithi120.stdout:44 files removed 2023-06-23T01:07:23.400 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-06-23T01:07:23.582 INFO:teuthology.orchestra.run.smithi134.stdout:Updating Subscription Management repositories. 2023-06-23T01:07:23.671 INFO:teuthology.orchestra.run.smithi157.stdout:Updating Subscription Management repositories. 2023-06-23T01:07:23.853 INFO:teuthology.orchestra.run.smithi120.stdout:Updating Subscription Management repositories. 2023-06-23T01:07:24.343 INFO:teuthology.orchestra.run.smithi134.stdout:ceph packages for x86_64 341 kB/s | 78 kB 00:00 2023-06-23T01:07:24.415 INFO:teuthology.orchestra.run.smithi157.stdout:ceph packages for x86_64 354 kB/s | 78 kB 00:00 2023-06-23T01:07:24.636 INFO:teuthology.orchestra.run.smithi134.stdout:ceph noarch packages 37 kB/s | 9.2 kB 00:00 2023-06-23T01:07:24.706 INFO:teuthology.orchestra.run.smithi157.stdout:ceph noarch packages 37 kB/s | 9.2 kB 00:00 2023-06-23T01:07:24.813 INFO:teuthology.orchestra.run.smithi120.stdout:ceph packages for x86_64 211 kB/s | 78 kB 00:00 2023-06-23T01:07:24.973 INFO:teuthology.orchestra.run.smithi134.stdout:ceph source packages 4.8 kB/s | 1.4 kB 00:00 2023-06-23T01:07:25.047 INFO:teuthology.orchestra.run.smithi157.stdout:ceph source packages 4.8 kB/s | 1.4 kB 00:00 2023-06-23T01:07:25.096 INFO:teuthology.orchestra.run.smithi120.stdout:ceph noarch packages 38 kB/s | 9.2 kB 00:00 2023-06-23T01:07:25.370 INFO:teuthology.orchestra.run.smithi120.stdout:ceph source packages 6.0 kB/s | 1.4 kB 00:00 2023-06-23T01:07:27.209 INFO:teuthology.orchestra.run.smithi134.stdout:Copr repo for python3-asyncssh owned by ceph 1.6 kB/s | 3.5 kB 00:02 2023-06-23T01:07:27.504 INFO:teuthology.orchestra.run.smithi157.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2023-06-23T01:07:27.617 INFO:teuthology.orchestra.run.smithi120.stdout:Copr repo for python3-asyncssh owned by ceph 1.6 kB/s | 3.5 kB 00:02 2023-06-23T01:07:28.071 INFO:teuthology.orchestra.run.smithi157.stdout:Extra Packages for Enterprise Linux 29 MB/s | 16 MB 00:00 2023-06-23T01:07:29.273 INFO:teuthology.orchestra.run.smithi134.stdout:Extra Packages for Enterprise Linux 7.7 MB/s | 16 MB 00:02 2023-06-23T01:07:30.399 INFO:teuthology.orchestra.run.smithi120.stdout:Extra Packages for Enterprise Linux 5.7 MB/s | 16 MB 00:02 2023-06-23T01:07:31.938 INFO:teuthology.orchestra.run.smithi157.stdout:lab-extras 4.2 MB/s | 24 kB 00:00 2023-06-23T01:07:32.352 INFO:teuthology.orchestra.run.smithi157.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 116 MB/s | 38 MB 00:00 2023-06-23T01:07:32.747 INFO:teuthology.orchestra.run.smithi134.stdout:lab-extras 4.4 MB/s | 24 kB 00:00 2023-06-23T01:07:33.121 INFO:teuthology.orchestra.run.smithi134.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 130 MB/s | 38 MB 00:00 2023-06-23T01:07:33.752 INFO:teuthology.orchestra.run.smithi120.stdout:lab-extras 3.8 MB/s | 24 kB 00:00 2023-06-23T01:07:34.218 INFO:teuthology.orchestra.run.smithi120.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 86 MB/s | 33 MB 00:00 2023-06-23T01:07:38.053 INFO:teuthology.orchestra.run.smithi157.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 119 MB/s | 33 MB 00:00 2023-06-23T01:07:38.630 INFO:teuthology.orchestra.run.smithi134.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 125 MB/s | 33 MB 00:00 2023-06-23T01:07:39.746 INFO:teuthology.orchestra.run.smithi120.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 90 MB/s | 38 MB 00:00 2023-06-23T01:07:46.259 INFO:teuthology.orchestra.run.smithi157.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-23T01:07:46.260 INFO:teuthology.orchestra.run.smithi157.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-23T01:07:46.270 INFO:teuthology.orchestra.run.smithi157.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-06-23T01:07:46.271 INFO:teuthology.orchestra.run.smithi157.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-06-23T01:07:46.272 INFO:teuthology.orchestra.run.smithi157.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-06-23T01:07:46.273 INFO:teuthology.orchestra.run.smithi157.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-06-23T01:07:46.275 INFO:teuthology.orchestra.run.smithi157.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-06-23T01:07:46.277 INFO:teuthology.orchestra.run.smithi157.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-06-23T01:07:46.278 INFO:teuthology.orchestra.run.smithi157.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-06-23T01:07:46.450 INFO:teuthology.orchestra.run.smithi134.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-23T01:07:46.451 INFO:teuthology.orchestra.run.smithi134.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-23T01:07:46.462 INFO:teuthology.orchestra.run.smithi134.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-06-23T01:07:46.463 INFO:teuthology.orchestra.run.smithi134.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-06-23T01:07:46.463 INFO:teuthology.orchestra.run.smithi134.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-06-23T01:07:46.464 INFO:teuthology.orchestra.run.smithi134.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-06-23T01:07:46.465 INFO:teuthology.orchestra.run.smithi134.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-06-23T01:07:46.468 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-06-23T01:07:46.469 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-06-23T01:07:46.766 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-06-23T01:07:46.777 INFO:teuthology.orchestra.run.smithi157.stdout:====================================================================================================================== 2023-06-23T01:07:46.778 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2023-06-23T01:07:46.778 INFO:teuthology.orchestra.run.smithi157.stdout:====================================================================================================================== 2023-06-23T01:07:46.778 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2023-06-23T01:07:46.778 INFO:teuthology.orchestra.run.smithi157.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-06-23T01:07:46.778 INFO:teuthology.orchestra.run.smithi157.stdout: ceph x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 6.4 k 2023-06-23T01:07:46.778 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 5.2 M 2023-06-23T01:07:46.779 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 873 k 2023-06-23T01:07:46.779 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 162 k 2023-06-23T01:07:46.779 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 1.6 M 2023-06-23T01:07:46.779 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 121 k 2023-06-23T01:07:46.779 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 1.7 M 2023-06-23T01:07:46.779 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 7.4 M 2023-06-23T01:07:46.779 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 47 k 2023-06-23T01:07:46.780 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 17 M 2023-06-23T01:07:46.780 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 52 M 2023-06-23T01:07:46.780 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 264 k 2023-06-23T01:07:46.780 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 80 k 2023-06-23T01:07:46.780 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-mirror x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 255 k 2023-06-23T01:07:46.780 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-top noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 18 k 2023-06-23T01:07:46.780 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-06-23T01:07:46.781 INFO:teuthology.orchestra.run.smithi157.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-06-23T01:07:46.781 INFO:teuthology.orchestra.run.smithi157.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-06-23T01:07:46.781 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 25 k 2023-06-23T01:07:46.781 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 794 k 2023-06-23T01:07:46.781 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 130 k 2023-06-23T01:07:46.781 INFO:teuthology.orchestra.run.smithi157.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-06-23T01:07:46.782 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-06-23T01:07:46.782 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-06-23T01:07:46.782 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 190 k 2023-06-23T01:07:46.782 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 371 k 2023-06-23T01:07:46.782 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 357 k 2023-06-23T01:07:46.783 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 107 k 2023-06-23T01:07:46.783 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 91 k 2023-06-23T01:07:46.783 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.5 M 2023-06-23T01:07:46.783 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 173 k 2023-06-23T01:07:46.783 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-06-23T01:07:46.783 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrading: 2023-06-23T01:07:46.784 INFO:teuthology.orchestra.run.smithi157.stdout: librados2 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.5 M 2023-06-23T01:07:46.784 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.6 M 2023-06-23T01:07:46.784 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2023-06-23T01:07:46.784 INFO:teuthology.orchestra.run.smithi157.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-06-23T01:07:46.784 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 23 M 2023-06-23T01:07:46.784 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 23 k 2023-06-23T01:07:46.784 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 2.3 M 2023-06-23T01:07:46.785 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 238 k 2023-06-23T01:07:46.785 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.9 M 2023-06-23T01:07:46.785 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 17 M 2023-06-23T01:07:46.785 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 14 k 2023-06-23T01:07:46.785 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 24 k 2023-06-23T01:07:46.785 INFO:teuthology.orchestra.run.smithi157.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-06-23T01:07:46.786 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-06-23T01:07:46.786 INFO:teuthology.orchestra.run.smithi157.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-06-23T01:07:46.786 INFO:teuthology.orchestra.run.smithi157.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-06-23T01:07:46.786 INFO:teuthology.orchestra.run.smithi157.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-06-23T01:07:46.786 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 174 k 2023-06-23T01:07:46.786 INFO:teuthology.orchestra.run.smithi157.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-06-23T01:07:46.786 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-06-23T01:07:46.787 INFO:teuthology.orchestra.run.smithi157.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-06-23T01:07:46.787 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-06-23T01:07:46.787 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-06-23T01:07:46.787 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 506 k 2023-06-23T01:07:46.787 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-06-23T01:07:46.787 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 5.5 M 2023-06-23T01:07:46.787 INFO:teuthology.orchestra.run.smithi157.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-06-23T01:07:46.787 INFO:teuthology.orchestra.run.smithi157.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-06-23T01:07:46.788 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-06-23T01:07:46.788 INFO:teuthology.orchestra.run.smithi157.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-06-23T01:07:46.788 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-06-23T01:07:46.788 INFO:teuthology.orchestra.run.smithi157.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-06-23T01:07:46.788 INFO:teuthology.orchestra.run.smithi157.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-06-23T01:07:46.788 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-06-23T01:07:46.788 INFO:teuthology.orchestra.run.smithi157.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-06-23T01:07:46.789 INFO:teuthology.orchestra.run.smithi157.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-06-23T01:07:46.789 INFO:teuthology.orchestra.run.smithi157.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-06-23T01:07:46.789 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-06-23T01:07:46.789 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-06-23T01:07:46.789 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-06-23T01:07:46.789 INFO:teuthology.orchestra.run.smithi157.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-06-23T01:07:46.789 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-06-23T01:07:46.790 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 45 k 2023-06-23T01:07:46.790 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 115 k 2023-06-23T01:07:46.790 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-06-23T01:07:46.790 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-06-23T01:07:46.790 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-06-23T01:07:46.790 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-06-23T01:07:46.790 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-06-23T01:07:46.790 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-06-23T01:07:46.791 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-06-23T01:07:46.791 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-06-23T01:07:46.791 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-06-23T01:07:46.791 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-23T01:07:46.791 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-06-23T01:07:46.791 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-06-23T01:07:46.791 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-06-23T01:07:46.791 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-06-23T01:07:46.792 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-06-23T01:07:46.792 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-06-23T01:07:46.792 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-06-23T01:07:46.792 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-06-23T01:07:46.792 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-06-23T01:07:46.792 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-06-23T01:07:46.793 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-06-23T01:07:46.793 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-06-23T01:07:46.793 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-06-23T01:07:46.793 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-06-23T01:07:46.793 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-06-23T01:07:46.793 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-06-23T01:07:46.793 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-23T01:07:46.793 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-06-23T01:07:46.794 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-06-23T01:07:46.794 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-06-23T01:07:46.794 INFO:teuthology.orchestra.run.smithi157.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-06-23T01:07:46.794 INFO:teuthology.orchestra.run.smithi157.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-06-23T01:07:46.794 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-06-23T01:07:46.794 INFO:teuthology.orchestra.run.smithi157.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-06-23T01:07:46.794 INFO:teuthology.orchestra.run.smithi157.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-06-23T01:07:46.794 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-06-23T01:07:46.795 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-06-23T01:07:46.795 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-06-23T01:07:46.795 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-06-23T01:07:46.795 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-06-23T01:07:46.795 INFO:teuthology.orchestra.run.smithi157.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-06-23T01:07:46.795 INFO:teuthology.orchestra.run.smithi157.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-06-23T01:07:46.795 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-06-23T01:07:46.796 INFO:teuthology.orchestra.run.smithi157.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-06-23T01:07:46.796 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-06-23T01:07:46.796 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2023-06-23T01:07:46.796 INFO:teuthology.orchestra.run.smithi157.stdout:====================================================================================================================== 2023-06-23T01:07:46.796 INFO:teuthology.orchestra.run.smithi157.stdout:Install 118 Packages 2023-06-23T01:07:46.796 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrade 2 Packages 2023-06-23T01:07:46.796 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-06-23T01:07:46.797 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 201 M 2023-06-23T01:07:46.797 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2023-06-23T01:07:46.955 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2023-06-23T01:07:46.966 INFO:teuthology.orchestra.run.smithi134.stdout:====================================================================================================================== 2023-06-23T01:07:46.966 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repository Size 2023-06-23T01:07:46.966 INFO:teuthology.orchestra.run.smithi134.stdout:====================================================================================================================== 2023-06-23T01:07:46.966 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2023-06-23T01:07:46.966 INFO:teuthology.orchestra.run.smithi134.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-06-23T01:07:46.967 INFO:teuthology.orchestra.run.smithi134.stdout: ceph x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 6.4 k 2023-06-23T01:07:46.967 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 5.2 M 2023-06-23T01:07:46.967 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 873 k 2023-06-23T01:07:46.967 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 162 k 2023-06-23T01:07:46.967 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 1.6 M 2023-06-23T01:07:46.967 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 121 k 2023-06-23T01:07:46.967 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 1.7 M 2023-06-23T01:07:46.968 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 7.4 M 2023-06-23T01:07:46.968 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 47 k 2023-06-23T01:07:46.968 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 17 M 2023-06-23T01:07:46.968 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 52 M 2023-06-23T01:07:46.968 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 264 k 2023-06-23T01:07:46.968 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 80 k 2023-06-23T01:07:46.969 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 255 k 2023-06-23T01:07:46.969 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 18 k 2023-06-23T01:07:46.969 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-06-23T01:07:46.969 INFO:teuthology.orchestra.run.smithi134.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-06-23T01:07:46.969 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-06-23T01:07:46.969 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 25 k 2023-06-23T01:07:46.969 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 794 k 2023-06-23T01:07:46.969 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 130 k 2023-06-23T01:07:46.970 INFO:teuthology.orchestra.run.smithi134.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-06-23T01:07:46.970 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-06-23T01:07:46.970 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-06-23T01:07:46.970 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 190 k 2023-06-23T01:07:46.970 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 371 k 2023-06-23T01:07:46.970 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 357 k 2023-06-23T01:07:46.970 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 107 k 2023-06-23T01:07:46.971 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 91 k 2023-06-23T01:07:46.971 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.5 M 2023-06-23T01:07:46.971 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 173 k 2023-06-23T01:07:46.971 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-06-23T01:07:46.971 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrading: 2023-06-23T01:07:46.971 INFO:teuthology.orchestra.run.smithi134.stdout: librados2 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.5 M 2023-06-23T01:07:46.972 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.6 M 2023-06-23T01:07:46.972 INFO:teuthology.orchestra.run.smithi134.stdout:Installing dependencies: 2023-06-23T01:07:46.972 INFO:teuthology.orchestra.run.smithi134.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-06-23T01:07:46.972 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 23 M 2023-06-23T01:07:46.972 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 23 k 2023-06-23T01:07:46.972 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 2.3 M 2023-06-23T01:07:46.972 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 238 k 2023-06-23T01:07:46.973 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.9 M 2023-06-23T01:07:46.973 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 17 M 2023-06-23T01:07:46.973 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 14 k 2023-06-23T01:07:46.973 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 24 k 2023-06-23T01:07:46.973 INFO:teuthology.orchestra.run.smithi134.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-06-23T01:07:46.973 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-06-23T01:07:46.973 INFO:teuthology.orchestra.run.smithi134.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-06-23T01:07:46.974 INFO:teuthology.orchestra.run.smithi134.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-06-23T01:07:46.974 INFO:teuthology.orchestra.run.smithi134.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-06-23T01:07:46.974 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 174 k 2023-06-23T01:07:46.974 INFO:teuthology.orchestra.run.smithi134.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-06-23T01:07:46.974 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-06-23T01:07:46.974 INFO:teuthology.orchestra.run.smithi134.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-06-23T01:07:46.974 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-06-23T01:07:46.975 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-06-23T01:07:46.975 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 506 k 2023-06-23T01:07:46.975 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-06-23T01:07:46.975 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 5.5 M 2023-06-23T01:07:46.975 INFO:teuthology.orchestra.run.smithi134.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-06-23T01:07:46.975 INFO:teuthology.orchestra.run.smithi134.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-06-23T01:07:46.976 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-06-23T01:07:46.976 INFO:teuthology.orchestra.run.smithi134.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-06-23T01:07:46.976 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-06-23T01:07:46.976 INFO:teuthology.orchestra.run.smithi134.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-06-23T01:07:46.976 INFO:teuthology.orchestra.run.smithi134.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-06-23T01:07:46.976 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-06-23T01:07:46.977 INFO:teuthology.orchestra.run.smithi134.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-06-23T01:07:46.977 INFO:teuthology.orchestra.run.smithi134.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-06-23T01:07:46.977 INFO:teuthology.orchestra.run.smithi134.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-06-23T01:07:46.977 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-06-23T01:07:46.977 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-06-23T01:07:46.977 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-06-23T01:07:46.977 INFO:teuthology.orchestra.run.smithi134.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-06-23T01:07:46.977 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-06-23T01:07:46.978 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 45 k 2023-06-23T01:07:46.978 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 115 k 2023-06-23T01:07:46.978 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-06-23T01:07:46.978 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-06-23T01:07:46.978 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-06-23T01:07:46.978 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-06-23T01:07:46.978 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-06-23T01:07:46.978 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-06-23T01:07:46.979 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-06-23T01:07:46.979 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-06-23T01:07:46.979 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-06-23T01:07:46.979 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-23T01:07:46.979 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-06-23T01:07:46.980 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-06-23T01:07:46.980 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-06-23T01:07:46.980 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-06-23T01:07:46.980 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-06-23T01:07:46.980 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-06-23T01:07:46.980 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-06-23T01:07:46.981 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-06-23T01:07:46.981 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-06-23T01:07:46.981 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-06-23T01:07:46.981 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-06-23T01:07:46.981 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-06-23T01:07:46.981 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-06-23T01:07:46.981 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-06-23T01:07:46.982 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-06-23T01:07:46.982 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-06-23T01:07:46.982 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-23T01:07:46.982 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-06-23T01:07:46.982 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-06-23T01:07:46.982 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-06-23T01:07:46.983 INFO:teuthology.orchestra.run.smithi134.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-06-23T01:07:46.983 INFO:teuthology.orchestra.run.smithi134.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-06-23T01:07:46.983 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-06-23T01:07:46.983 INFO:teuthology.orchestra.run.smithi134.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-06-23T01:07:46.983 INFO:teuthology.orchestra.run.smithi134.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-06-23T01:07:46.983 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-06-23T01:07:46.984 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-06-23T01:07:46.984 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-06-23T01:07:46.984 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-06-23T01:07:46.984 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-06-23T01:07:46.984 INFO:teuthology.orchestra.run.smithi134.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-06-23T01:07:46.984 INFO:teuthology.orchestra.run.smithi134.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-06-23T01:07:46.985 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-06-23T01:07:46.985 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-06-23T01:07:46.985 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-06-23T01:07:46.985 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2023-06-23T01:07:46.985 INFO:teuthology.orchestra.run.smithi134.stdout:====================================================================================================================== 2023-06-23T01:07:46.985 INFO:teuthology.orchestra.run.smithi134.stdout:Install 118 Packages 2023-06-23T01:07:46.985 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrade 2 Packages 2023-06-23T01:07:46.985 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-06-23T01:07:46.986 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 201 M 2023-06-23T01:07:46.986 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2023-06-23T01:07:47.088 INFO:teuthology.orchestra.run.smithi157.stdout:(1/120): ceph-17.2.6-719.g42ddc46e.el8.x86_64.r 43 kB/s | 6.4 kB 00:00 2023-06-23T01:07:47.288 INFO:teuthology.orchestra.run.smithi134.stdout:(1/120): ceph-17.2.6-719.g42ddc46e.el8.x86_64.r 38 kB/s | 6.4 kB 00:00 2023-06-23T01:07:47.363 INFO:teuthology.orchestra.run.smithi157.stdout:(2/120): ceph-fuse-17.2.6-719.g42ddc46e.el8.x86 3.1 MB/s | 873 kB 00:00 2023-06-23T01:07:47.419 INFO:teuthology.orchestra.run.smithi157.stdout:(3/120): ceph-immutable-object-cache-17.2.6-719 2.9 MB/s | 162 kB 00:00 2023-06-23T01:07:47.859 INFO:teuthology.orchestra.run.smithi157.stdout:(4/120): ceph-mds-17.2.6-719.g42ddc46e.el8.x86_ 5.2 MB/s | 2.3 MB 00:00 2023-06-23T01:07:47.976 INFO:teuthology.orchestra.run.smithi157.stdout:(5/120): ceph-base-17.2.6-719.g42ddc46e.el8.x86 5.0 MB/s | 5.2 MB 00:01 2023-06-23T01:07:48.228 INFO:teuthology.orchestra.run.smithi157.stdout:(6/120): ceph-mgr-17.2.6-719.g42ddc46e.el8.x86_ 4.3 MB/s | 1.6 MB 00:00 2023-06-23T01:07:48.287 INFO:teuthology.orchestra.run.smithi134.stdout:(2/120): ceph-fuse-17.2.6-719.g42ddc46e.el8.x86 874 kB/s | 873 kB 00:00 2023-06-23T01:07:48.385 INFO:teuthology.orchestra.run.smithi120.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-23T01:07:48.386 INFO:teuthology.orchestra.run.smithi120.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-23T01:07:48.397 INFO:teuthology.orchestra.run.smithi120.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-06-23T01:07:48.398 INFO:teuthology.orchestra.run.smithi120.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-06-23T01:07:48.399 INFO:teuthology.orchestra.run.smithi120.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-06-23T01:07:48.400 INFO:teuthology.orchestra.run.smithi120.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-06-23T01:07:48.401 INFO:teuthology.orchestra.run.smithi120.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-06-23T01:07:48.404 INFO:teuthology.orchestra.run.smithi120.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-06-23T01:07:48.405 INFO:teuthology.orchestra.run.smithi120.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-06-23T01:07:48.459 INFO:teuthology.orchestra.run.smithi134.stdout:(3/120): ceph-immutable-object-cache-17.2.6-719 947 kB/s | 162 kB 00:00 2023-06-23T01:07:48.533 INFO:teuthology.orchestra.run.smithi157.stdout:(7/120): ceph-mon-17.2.6-719.g42ddc46e.el8.x86_ 7.1 MB/s | 3.9 MB 00:00 2023-06-23T01:07:48.700 INFO:teuthology.orchestra.run.smithi157.stdout:(8/120): ceph-common-17.2.6-719.g42ddc46e.el8.x 13 MB/s | 23 MB 00:01 2023-06-23T01:07:48.729 INFO:teuthology.orchestra.run.smithi157.stdout:(9/120): ceph-selinux-17.2.6-719.g42ddc46e.el8. 839 kB/s | 24 kB 00:00 2023-06-23T01:07:48.907 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2023-06-23T01:07:48.919 INFO:teuthology.orchestra.run.smithi120.stdout:====================================================================================================================== 2023-06-23T01:07:48.920 INFO:teuthology.orchestra.run.smithi120.stdout: Package Arch Version Repository Size 2023-06-23T01:07:48.920 INFO:teuthology.orchestra.run.smithi120.stdout:====================================================================================================================== 2023-06-23T01:07:48.920 INFO:teuthology.orchestra.run.smithi120.stdout:Installing: 2023-06-23T01:07:48.920 INFO:teuthology.orchestra.run.smithi120.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-06-23T01:07:48.920 INFO:teuthology.orchestra.run.smithi120.stdout: ceph x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 6.4 k 2023-06-23T01:07:48.921 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-base x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 5.2 M 2023-06-23T01:07:48.921 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-fuse x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 873 k 2023-06-23T01:07:48.921 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 162 k 2023-06-23T01:07:48.921 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 1.6 M 2023-06-23T01:07:48.921 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-cephadm noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 121 k 2023-06-23T01:07:48.922 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-dashboard noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 1.7 M 2023-06-23T01:07:48.922 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 7.4 M 2023-06-23T01:07:48.922 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-rook noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 47 k 2023-06-23T01:07:48.922 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-radosgw x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 17 M 2023-06-23T01:07:48.922 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-test x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 52 M 2023-06-23T01:07:48.922 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-volume noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 264 k 2023-06-23T01:07:48.923 INFO:teuthology.orchestra.run.smithi120.stdout: cephadm noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 80 k 2023-06-23T01:07:48.923 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-mirror x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 255 k 2023-06-23T01:07:48.923 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-top noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 18 k 2023-06-23T01:07:48.924 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-06-23T01:07:48.924 INFO:teuthology.orchestra.run.smithi120.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-06-23T01:07:48.924 INFO:teuthology.orchestra.run.smithi120.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-06-23T01:07:48.925 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs-devel x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 25 k 2023-06-23T01:07:48.925 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs2 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 794 k 2023-06-23T01:07:48.925 INFO:teuthology.orchestra.run.smithi120.stdout: librados-devel x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 130 k 2023-06-23T01:07:48.925 INFO:teuthology.orchestra.run.smithi120.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-06-23T01:07:48.925 INFO:teuthology.orchestra.run.smithi120.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-06-23T01:07:48.925 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-06-23T01:07:48.926 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cephfs x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 190 k 2023-06-23T01:07:48.926 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rados x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 371 k 2023-06-23T01:07:48.926 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rbd x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 357 k 2023-06-23T01:07:48.926 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rgw x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 107 k 2023-06-23T01:07:48.926 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-fuse x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 91 k 2023-06-23T01:07:48.926 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-mirror x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.5 M 2023-06-23T01:07:48.927 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-nbd x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 173 k 2023-06-23T01:07:48.928 INFO:teuthology.orchestra.run.smithi120.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-06-23T01:07:48.928 INFO:teuthology.orchestra.run.smithi120.stdout:Upgrading: 2023-06-23T01:07:48.928 INFO:teuthology.orchestra.run.smithi120.stdout: librados2 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.5 M 2023-06-23T01:07:48.929 INFO:teuthology.orchestra.run.smithi120.stdout: librbd1 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.6 M 2023-06-23T01:07:48.929 INFO:teuthology.orchestra.run.smithi120.stdout:Installing dependencies: 2023-06-23T01:07:48.929 INFO:teuthology.orchestra.run.smithi120.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-06-23T01:07:48.930 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-common x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 23 M 2023-06-23T01:07:48.930 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-grafana-dashboards noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 23 k 2023-06-23T01:07:48.930 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mds x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 2.3 M 2023-06-23T01:07:48.930 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-modules-core noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 238 k 2023-06-23T01:07:48.931 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mon x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.9 M 2023-06-23T01:07:48.931 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-osd x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 17 M 2023-06-23T01:07:48.931 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-prometheus-alerts noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 14 k 2023-06-23T01:07:48.932 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-selinux x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 24 k 2023-06-23T01:07:48.932 INFO:teuthology.orchestra.run.smithi120.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-06-23T01:07:48.932 INFO:teuthology.orchestra.run.smithi120.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-06-23T01:07:48.933 INFO:teuthology.orchestra.run.smithi120.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-06-23T01:07:48.933 INFO:teuthology.orchestra.run.smithi120.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-06-23T01:07:48.933 INFO:teuthology.orchestra.run.smithi120.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-06-23T01:07:48.934 INFO:teuthology.orchestra.run.smithi120.stdout: libcephsqlite x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 174 k 2023-06-23T01:07:48.934 INFO:teuthology.orchestra.run.smithi120.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-06-23T01:07:48.934 INFO:teuthology.orchestra.run.smithi120.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-06-23T01:07:48.934 INFO:teuthology.orchestra.run.smithi120.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-06-23T01:07:48.935 INFO:teuthology.orchestra.run.smithi120.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-06-23T01:07:48.935 INFO:teuthology.orchestra.run.smithi120.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-06-23T01:07:48.935 INFO:teuthology.orchestra.run.smithi120.stdout: libradosstriper1 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 506 k 2023-06-23T01:07:48.935 INFO:teuthology.orchestra.run.smithi120.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-06-23T01:07:48.935 INFO:teuthology.orchestra.run.smithi120.stdout: librgw2 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 5.5 M 2023-06-23T01:07:48.936 INFO:teuthology.orchestra.run.smithi120.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-06-23T01:07:48.936 INFO:teuthology.orchestra.run.smithi120.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-06-23T01:07:48.936 INFO:teuthology.orchestra.run.smithi120.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-06-23T01:07:48.937 INFO:teuthology.orchestra.run.smithi120.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-06-23T01:07:48.937 INFO:teuthology.orchestra.run.smithi120.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-06-23T01:07:48.937 INFO:teuthology.orchestra.run.smithi120.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-06-23T01:07:48.937 INFO:teuthology.orchestra.run.smithi120.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-06-23T01:07:48.938 INFO:teuthology.orchestra.run.smithi120.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-06-23T01:07:48.938 INFO:teuthology.orchestra.run.smithi120.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-06-23T01:07:48.938 INFO:teuthology.orchestra.run.smithi120.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-06-23T01:07:48.939 INFO:teuthology.orchestra.run.smithi120.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-06-23T01:07:48.939 INFO:teuthology.orchestra.run.smithi120.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-06-23T01:07:48.939 INFO:teuthology.orchestra.run.smithi120.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-06-23T01:07:48.940 INFO:teuthology.orchestra.run.smithi120.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-06-23T01:07:48.940 INFO:teuthology.orchestra.run.smithi120.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-06-23T01:07:48.940 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-06-23T01:07:48.940 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-argparse x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 45 k 2023-06-23T01:07:48.940 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-common x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 115 k 2023-06-23T01:07:48.940 INFO:teuthology.orchestra.run.smithi120.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-06-23T01:07:48.941 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-06-23T01:07:48.941 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-06-23T01:07:48.941 INFO:teuthology.orchestra.run.smithi120.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-06-23T01:07:48.941 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-06-23T01:07:48.941 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-06-23T01:07:48.942 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-06-23T01:07:48.942 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-06-23T01:07:48.942 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-06-23T01:07:48.942 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-23T01:07:48.942 INFO:teuthology.orchestra.run.smithi120.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-06-23T01:07:48.942 INFO:teuthology.orchestra.run.smithi120.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-06-23T01:07:48.942 INFO:teuthology.orchestra.run.smithi120.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-06-23T01:07:48.943 INFO:teuthology.orchestra.run.smithi120.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-06-23T01:07:48.943 INFO:teuthology.orchestra.run.smithi120.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-06-23T01:07:48.943 INFO:teuthology.orchestra.run.smithi120.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-06-23T01:07:48.943 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-06-23T01:07:48.943 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-06-23T01:07:48.943 INFO:teuthology.orchestra.run.smithi120.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-06-23T01:07:48.944 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-06-23T01:07:48.944 INFO:teuthology.orchestra.run.smithi120.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-06-23T01:07:48.944 INFO:teuthology.orchestra.run.smithi120.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-06-23T01:07:48.944 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-06-23T01:07:48.944 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-06-23T01:07:48.944 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-06-23T01:07:48.945 INFO:teuthology.orchestra.run.smithi120.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-06-23T01:07:48.945 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-23T01:07:48.945 INFO:teuthology.orchestra.run.smithi120.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-06-23T01:07:48.945 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-06-23T01:07:48.945 INFO:teuthology.orchestra.run.smithi120.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-06-23T01:07:48.945 INFO:teuthology.orchestra.run.smithi120.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-06-23T01:07:48.945 INFO:teuthology.orchestra.run.smithi120.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-06-23T01:07:48.946 INFO:teuthology.orchestra.run.smithi120.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-06-23T01:07:48.946 INFO:teuthology.orchestra.run.smithi120.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-06-23T01:07:48.946 INFO:teuthology.orchestra.run.smithi120.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-06-23T01:07:48.946 INFO:teuthology.orchestra.run.smithi120.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-06-23T01:07:48.946 INFO:teuthology.orchestra.run.smithi120.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-06-23T01:07:48.946 INFO:teuthology.orchestra.run.smithi120.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-06-23T01:07:48.946 INFO:teuthology.orchestra.run.smithi120.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-06-23T01:07:48.947 INFO:teuthology.orchestra.run.smithi120.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-06-23T01:07:48.947 INFO:teuthology.orchestra.run.smithi120.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-06-23T01:07:48.947 INFO:teuthology.orchestra.run.smithi120.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-06-23T01:07:48.947 INFO:teuthology.orchestra.run.smithi120.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-06-23T01:07:48.947 INFO:teuthology.orchestra.run.smithi120.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-06-23T01:07:48.947 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:07:48.947 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction Summary 2023-06-23T01:07:48.948 INFO:teuthology.orchestra.run.smithi120.stdout:====================================================================================================================== 2023-06-23T01:07:48.948 INFO:teuthology.orchestra.run.smithi120.stdout:Install 118 Packages 2023-06-23T01:07:48.948 INFO:teuthology.orchestra.run.smithi120.stdout:Upgrade 2 Packages 2023-06-23T01:07:48.948 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:07:48.949 INFO:teuthology.orchestra.run.smithi120.stdout:Total download size: 201 M 2023-06-23T01:07:48.949 INFO:teuthology.orchestra.run.smithi120.stdout:Downloading Packages: 2023-06-23T01:07:49.392 INFO:teuthology.orchestra.run.smithi134.stdout:(4/120): ceph-mds-17.2.6-719.g42ddc46e.el8.x86_ 2.5 MB/s | 2.3 MB 00:00 2023-06-23T01:07:49.539 INFO:teuthology.orchestra.run.smithi120.stdout:(1/120): ceph-17.2.6-719.g42ddc46e.el8.x86_64.r 29 kB/s | 6.4 kB 00:00 2023-06-23T01:07:49.923 INFO:teuthology.orchestra.run.smithi134.stdout:(5/120): ceph-mgr-17.2.6-719.g42ddc46e.el8.x86_ 3.0 MB/s | 1.6 MB 00:00 2023-06-23T01:07:50.294 INFO:teuthology.orchestra.run.smithi120.stdout:(2/120): ceph-fuse-17.2.6-719.g42ddc46e.el8.x86 1.1 MB/s | 873 kB 00:00 2023-06-23T01:07:50.351 INFO:teuthology.orchestra.run.smithi157.stdout:(10/120): ceph-radosgw-17.2.6-719.g42ddc46e.el8 9.2 MB/s | 17 MB 00:01 2023-06-23T01:07:50.370 INFO:teuthology.orchestra.run.smithi120.stdout:(3/120): ceph-immutable-object-cache-17.2.6-719 2.1 MB/s | 162 kB 00:00 2023-06-23T01:07:50.460 INFO:teuthology.orchestra.run.smithi157.stdout:(11/120): cephfs-mirror-17.2.6-719.g42ddc46e.el 2.3 MB/s | 255 kB 00:00 2023-06-23T01:07:50.569 INFO:teuthology.orchestra.run.smithi157.stdout:(12/120): libcephfs-devel-17.2.6-719.g42ddc46e. 228 kB/s | 25 kB 00:00 2023-06-23T01:07:50.806 INFO:teuthology.orchestra.run.smithi134.stdout:(6/120): ceph-mon-17.2.6-719.g42ddc46e.el8.x86_ 4.5 MB/s | 3.9 MB 00:00 2023-06-23T01:07:51.042 INFO:teuthology.orchestra.run.smithi157.stdout:(13/120): libcephfs2-17.2.6-719.g42ddc46e.el8.x 1.6 MB/s | 794 kB 00:00 2023-06-23T01:07:51.307 INFO:teuthology.orchestra.run.smithi157.stdout:(14/120): libcephsqlite-17.2.6-719.g42ddc46e.el 656 kB/s | 174 kB 00:00 2023-06-23T01:07:51.363 INFO:teuthology.orchestra.run.smithi134.stdout:(7/120): ceph-base-17.2.6-719.g42ddc46e.el8.x86 1.2 MB/s | 5.2 MB 00:04 2023-06-23T01:07:51.446 INFO:teuthology.orchestra.run.smithi157.stdout:(15/120): librados-devel-17.2.6-719.g42ddc46e.e 936 kB/s | 130 kB 00:00 2023-06-23T01:07:51.476 INFO:teuthology.orchestra.run.smithi120.stdout:(4/120): ceph-mds-17.2.6-719.g42ddc46e.el8.x86_ 2.1 MB/s | 2.3 MB 00:01 2023-06-23T01:07:51.823 INFO:teuthology.orchestra.run.smithi157.stdout:(16/120): libradosstriper1-17.2.6-719.g42ddc46e 1.3 MB/s | 506 kB 00:00 2023-06-23T01:07:52.100 INFO:teuthology.orchestra.run.smithi120.stdout:(5/120): ceph-base-17.2.6-719.g42ddc46e.el8.x86 1.9 MB/s | 5.2 MB 00:02 2023-06-23T01:07:52.328 INFO:teuthology.orchestra.run.smithi120.stdout:(6/120): ceph-mgr-17.2.6-719.g42ddc46e.el8.x86_ 1.8 MB/s | 1.6 MB 00:00 2023-06-23T01:07:53.026 INFO:teuthology.orchestra.run.smithi157.stdout:(17/120): ceph-osd-17.2.6-719.g42ddc46e.el8.x86 3.5 MB/s | 17 MB 00:04 2023-06-23T01:07:53.099 INFO:teuthology.orchestra.run.smithi157.stdout:(18/120): python3-ceph-argparse-17.2.6-719.g42d 610 kB/s | 45 kB 00:00 2023-06-23T01:07:53.179 INFO:teuthology.orchestra.run.smithi157.stdout:(19/120): python3-ceph-common-17.2.6-719.g42ddc 1.4 MB/s | 115 kB 00:00 2023-06-23T01:07:53.262 INFO:teuthology.orchestra.run.smithi157.stdout:(20/120): python3-cephfs-17.2.6-719.g42ddc46e.e 2.2 MB/s | 190 kB 00:00 2023-06-23T01:07:53.425 INFO:teuthology.orchestra.run.smithi157.stdout:(21/120): python3-rados-17.2.6-719.g42ddc46e.el 2.2 MB/s | 371 kB 00:00 2023-06-23T01:07:53.591 INFO:teuthology.orchestra.run.smithi157.stdout:(22/120): python3-rbd-17.2.6-719.g42ddc46e.el8. 2.1 MB/s | 357 kB 00:00 2023-06-23T01:07:53.677 INFO:teuthology.orchestra.run.smithi157.stdout:(23/120): python3-rgw-17.2.6-719.g42ddc46e.el8. 1.2 MB/s | 107 kB 00:00 2023-06-23T01:07:53.761 INFO:teuthology.orchestra.run.smithi157.stdout:(24/120): rbd-fuse-17.2.6-719.g42ddc46e.el8.x86 1.1 MB/s | 91 kB 00:00 2023-06-23T01:07:53.898 INFO:teuthology.orchestra.run.smithi120.stdout:(7/120): ceph-mon-17.2.6-719.g42ddc46e.el8.x86_ 2.2 MB/s | 3.9 MB 00:01 2023-06-23T01:07:54.017 INFO:teuthology.orchestra.run.smithi120.stdout:(8/120): ceph-common-17.2.6-719.g42ddc46e.el8.x 5.0 MB/s | 23 MB 00:04 2023-06-23T01:07:54.070 INFO:teuthology.orchestra.run.smithi120.stdout:(9/120): ceph-selinux-17.2.6-719.g42ddc46e.el8. 466 kB/s | 24 kB 00:00 2023-06-23T01:07:54.446 INFO:teuthology.orchestra.run.smithi157.stdout:(25/120): librgw2-17.2.6-719.g42ddc46e.el8.x86_ 2.1 MB/s | 5.5 MB 00:02 2023-06-23T01:07:54.529 INFO:teuthology.orchestra.run.smithi157.stdout:(26/120): rbd-nbd-17.2.6-719.g42ddc46e.el8.x86_ 2.0 MB/s | 173 kB 00:00 2023-06-23T01:07:54.607 INFO:teuthology.orchestra.run.smithi157.stdout:(27/120): ceph-grafana-dashboards-17.2.6-719.g4 296 kB/s | 23 kB 00:00 2023-06-23T01:07:54.686 INFO:teuthology.orchestra.run.smithi157.stdout:(28/120): ceph-mgr-cephadm-17.2.6-719.g42ddc46e 1.5 MB/s | 121 kB 00:00 2023-06-23T01:07:54.741 INFO:teuthology.orchestra.run.smithi134.stdout:(8/120): ceph-osd-17.2.6-719.g42ddc46e.el8.x86_ 4.2 MB/s | 17 MB 00:03 2023-06-23T01:07:54.808 INFO:teuthology.orchestra.run.smithi157.stdout:(29/120): rbd-mirror-17.2.6-719.g42ddc46e.el8.x 3.3 MB/s | 3.5 MB 00:01 2023-06-23T01:07:54.813 INFO:teuthology.orchestra.run.smithi134.stdout:(9/120): ceph-selinux-17.2.6-719.g42ddc46e.el8. 338 kB/s | 24 kB 00:00 2023-06-23T01:07:55.439 INFO:teuthology.orchestra.run.smithi157.stdout:(30/120): ceph-mgr-dashboard-17.2.6-719.g42ddc4 2.3 MB/s | 1.7 MB 00:00 2023-06-23T01:07:55.607 INFO:teuthology.orchestra.run.smithi157.stdout:(31/120): ceph-mgr-modules-core-17.2.6-719.g42d 1.4 MB/s | 238 kB 00:00 2023-06-23T01:07:55.695 INFO:teuthology.orchestra.run.smithi157.stdout:(32/120): ceph-mgr-rook-17.2.6-719.g42ddc46e.el 544 kB/s | 47 kB 00:00 2023-06-23T01:07:55.772 INFO:teuthology.orchestra.run.smithi157.stdout:(33/120): ceph-prometheus-alerts-17.2.6-719.g42 186 kB/s | 14 kB 00:00 2023-06-23T01:07:56.291 INFO:teuthology.orchestra.run.smithi157.stdout:(34/120): ceph-test-17.2.6-719.g42ddc46e.el8.x8 6.8 MB/s | 52 MB 00:07 2023-06-23T01:07:56.294 INFO:teuthology.orchestra.run.smithi157.stdout:(35/120): ceph-volume-17.2.6-719.g42ddc46e.el8. 506 kB/s | 264 kB 00:00 2023-06-23T01:07:56.348 INFO:teuthology.orchestra.run.smithi157.stdout:(36/120): cephadm-17.2.6-719.g42ddc46e.el8.noar 1.4 MB/s | 80 kB 00:00 2023-06-23T01:07:56.365 INFO:teuthology.orchestra.run.smithi157.stdout:[MIRROR] gperftools-libs-2.7-9.el8.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/g/gperftools-libs-2.7-9.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T01:07:56.395 INFO:teuthology.orchestra.run.smithi157.stdout:(37/120): cephfs-top-17.2.6-719.g42ddc46e.el8.n 176 kB/s | 18 kB 00:00 2023-06-23T01:07:56.396 INFO:teuthology.orchestra.run.smithi157.stdout:[MIRROR] liboath-2.6.2-3.el8.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/l/liboath-2.6.2-3.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T01:07:56.398 INFO:teuthology.orchestra.run.smithi157.stdout:[MIRROR] gperftools-libs-2.7-9.el8.x86_64.rpm: Status code: 404 for http://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/g/gperftools-libs-2.7-9.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T01:07:56.398 INFO:teuthology.orchestra.run.smithi157.stdout:[MIRROR] liboath-2.6.2-3.el8.x86_64.rpm: Status code: 404 for http://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/l/liboath-2.6.2-3.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T01:07:56.505 INFO:teuthology.orchestra.run.smithi157.stdout:[MIRROR] gperftools-libs-2.7-9.el8.x86_64.rpm: Status code: 403 for https://download-ib01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/g/gperftools-libs-2.7-9.el8.x86_64.rpm (IP: 152.19.134.145) 2023-06-23T01:07:56.512 INFO:teuthology.orchestra.run.smithi157.stdout:[MIRROR] liboath-2.6.2-3.el8.x86_64.rpm: Status code: 403 for https://download-ib01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/l/liboath-2.6.2-3.el8.x86_64.rpm (IP: 152.19.134.145) 2023-06-23T01:07:56.564 INFO:teuthology.orchestra.run.smithi157.stdout:[MIRROR] gperftools-libs-2.7-9.el8.x86_64.rpm: Status code: 403 for http://download-ib01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/g/gperftools-libs-2.7-9.el8.x86_64.rpm (IP: 152.19.134.145) 2023-06-23T01:07:56.567 INFO:teuthology.orchestra.run.smithi157.stdout:[MIRROR] liboath-2.6.2-3.el8.x86_64.rpm: Status code: 403 for http://download-ib01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/l/liboath-2.6.2-3.el8.x86_64.rpm (IP: 152.19.134.145) 2023-06-23T01:07:56.847 INFO:teuthology.orchestra.run.smithi157.stdout:(38/120): liboath-2.6.2-3.el8.x86_64.rpm 131 kB/s | 59 kB 00:00 2023-06-23T01:07:56.847 INFO:teuthology.orchestra.run.smithi157.stdout:[MIRROR] libunwind-1.3.1-3.el8.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/l/libunwind-1.3.1-3.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T01:07:56.848 INFO:teuthology.orchestra.run.smithi157.stdout:[MIRROR] libunwind-1.3.1-3.el8.x86_64.rpm: Status code: 404 for http://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/l/libunwind-1.3.1-3.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T01:07:56.875 INFO:teuthology.orchestra.run.smithi157.stdout:[MIRROR] libunwind-1.3.1-3.el8.x86_64.rpm: Status code: 403 for https://download-ib01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/l/libunwind-1.3.1-3.el8.x86_64.rpm (IP: 152.19.134.145) 2023-06-23T01:07:56.904 INFO:teuthology.orchestra.run.smithi157.stdout:[MIRROR] libunwind-1.3.1-3.el8.x86_64.rpm: Status code: 403 for http://download-ib01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/l/libunwind-1.3.1-3.el8.x86_64.rpm (IP: 152.19.134.145) 2023-06-23T01:07:56.986 INFO:teuthology.orchestra.run.smithi157.stdout:(39/120): gperftools-libs-2.7-9.el8.x86_64.rpm 480 kB/s | 306 kB 00:00 2023-06-23T01:07:56.987 INFO:teuthology.orchestra.run.smithi157.stdout:[MIRROR] python3-asyncssh-2.7.0-2.el8.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/p/python3-asyncssh-2.7.0-2.el8.noarch.rpm (IP: 8.43.85.72) 2023-06-23T01:07:57.014 INFO:teuthology.orchestra.run.smithi157.stdout:[MIRROR] python3-asyncssh-2.7.0-2.el8.noarch.rpm: Status code: 403 for https://download-ib01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/p/python3-asyncssh-2.7.0-2.el8.noarch.rpm (IP: 152.19.134.145) 2023-06-23T01:07:57.015 INFO:teuthology.orchestra.run.smithi157.stdout:[MIRROR] python3-asyncssh-2.7.0-2.el8.noarch.rpm: Status code: 404 for http://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/p/python3-asyncssh-2.7.0-2.el8.noarch.rpm (IP: 8.43.85.72) 2023-06-23T01:07:57.044 INFO:teuthology.orchestra.run.smithi157.stdout:(40/120): libunwind-1.3.1-3.el8.x86_64.rpm 379 kB/s | 75 kB 00:00 2023-06-23T01:07:57.115 INFO:teuthology.orchestra.run.smithi157.stdout:(41/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 617 kB/s | 44 kB 00:00 2023-06-23T01:07:57.140 INFO:teuthology.orchestra.run.smithi157.stdout:(42/120): ceph-mgr-diskprediction-local-17.2.6- 3.2 MB/s | 7.4 MB 00:02 2023-06-23T01:07:57.159 INFO:teuthology.orchestra.run.smithi157.stdout:(43/120): python3-asyncssh-2.7.0-2.el8.noarch.r 2.7 MB/s | 476 kB 00:00 2023-06-23T01:07:57.231 INFO:teuthology.orchestra.run.smithi157.stdout:(44/120): python3-certifi-2018.10.15-7.el8.noar 231 kB/s | 16 kB 00:00 2023-06-23T01:07:57.255 INFO:teuthology.orchestra.run.smithi157.stdout:(45/120): python3-beautifulsoup4-4.6.3-2.el8.1. 1.3 MB/s | 185 kB 00:00 2023-06-23T01:07:57.303 INFO:teuthology.orchestra.run.smithi157.stdout:(46/120): python3-cheroot-8.5.2-1.el8.noarch.rp 2.3 MB/s | 173 kB 00:00 2023-06-23T01:07:57.351 INFO:teuthology.orchestra.run.smithi157.stdout:(47/120): python3-cachetools-3.1.1-4.el8.noarch 157 kB/s | 33 kB 00:00 2023-06-23T01:07:57.374 INFO:teuthology.orchestra.run.smithi157.stdout:(48/120): python3-google-auth-1.1.1-10.el8.noar 1.1 MB/s | 81 kB 00:00 2023-06-23T01:07:57.396 INFO:teuthology.orchestra.run.smithi157.stdout:(49/120): python3-cherrypy-18.4.0-1.el8.noarch. 2.7 MB/s | 384 kB 00:00 2023-06-23T01:07:57.421 INFO:teuthology.orchestra.run.smithi157.stdout:(50/120): python3-jaraco-6.2-6.el8.noarch.rpm 158 kB/s | 11 kB 00:00 2023-06-23T01:07:57.445 INFO:teuthology.orchestra.run.smithi157.stdout:(51/120): python3-jaraco-functools-2.0-4.el8.no 249 kB/s | 18 kB 00:00 2023-06-23T01:07:57.516 INFO:teuthology.orchestra.run.smithi157.stdout:(52/120): python3-more-itertools-7.2.0-3.el8.no 828 kB/s | 59 kB 00:00 2023-06-23T01:07:57.543 INFO:teuthology.orchestra.run.smithi157.stdout:(53/120): python3-kubernetes-11.0.0-6.el8.noarc 8.0 MB/s | 1.2 MB 00:00 2023-06-23T01:07:57.562 INFO:teuthology.orchestra.run.smithi157.stdout:(54/120): python3-logutils-0.3.5-11.el8.noarch. 348 kB/s | 49 kB 00:00 2023-06-23T01:07:57.587 INFO:teuthology.orchestra.run.smithi157.stdout:(55/120): python3-natsort-7.1.1-2.el8.noarch.rp 851 kB/s | 60 kB 00:00 2023-06-23T01:07:57.617 INFO:teuthology.orchestra.run.smithi157.stdout:(56/120): python3-pecan-1.3.2-9.el8.noarch.rpm 3.7 MB/s | 283 kB 00:00 2023-06-23T01:07:57.632 INFO:teuthology.orchestra.run.smithi157.stdout:(57/120): python3-portend-2.6-1.el8.noarch.rpm 222 kB/s | 16 kB 00:00 2023-06-23T01:07:57.658 INFO:teuthology.orchestra.run.smithi157.stdout:(58/120): python3-repoze-lru-0.7-6.el8.noarch.r 477 kB/s | 34 kB 00:00 2023-06-23T01:07:57.691 INFO:teuthology.orchestra.run.smithi157.stdout:(59/120): python3-routes-2.4.1-12.el8.noarch.rp 2.6 MB/s | 196 kB 00:00 2023-06-23T01:07:57.703 INFO:teuthology.orchestra.run.smithi157.stdout:(60/120): python3-rsa-4.8-1.el8.noarch.rpm 868 kB/s | 61 kB 00:00 2023-06-23T01:07:57.729 INFO:teuthology.orchestra.run.smithi157.stdout:(61/120): python3-simplegeneric-0.8.1-17.el8.no 266 kB/s | 19 kB 00:00 2023-06-23T01:07:57.761 INFO:teuthology.orchestra.run.smithi157.stdout:(62/120): python3-singledispatch-3.4.0.3-18.el8 338 kB/s | 24 kB 00:00 2023-06-23T01:07:57.776 INFO:teuthology.orchestra.run.smithi157.stdout:(63/120): python3-tempora-1.14.1-5.el8.noarch.r 414 kB/s | 29 kB 00:00 2023-06-23T01:07:57.799 INFO:teuthology.orchestra.run.smithi157.stdout:(64/120): python3-trustme-0.6.0-4.el8.noarch.rp 390 kB/s | 27 kB 00:00 2023-06-23T01:07:57.835 INFO:teuthology.orchestra.run.smithi157.stdout:(65/120): python3-waitress-1.4.3-1.el8.noarch.r 3.4 MB/s | 255 kB 00:00 2023-06-23T01:07:57.871 INFO:teuthology.orchestra.run.smithi157.stdout:(66/120): python3-websocket-client-0.56.0-5.el8 863 kB/s | 61 kB 00:00 2023-06-23T01:07:57.906 INFO:teuthology.orchestra.run.smithi157.stdout:(67/120): python3-webtest-2.0.33-1.el8.noarch.r 1.2 MB/s | 85 kB 00:00 2023-06-23T01:07:57.915 INFO:teuthology.orchestra.run.smithi157.stdout:(68/120): python3-webob-1.8.5-1.el8.1.noarch.rp 1.7 MB/s | 251 kB 00:00 2023-06-23T01:07:57.923 INFO:teuthology.orchestra.run.smithi157.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 9.5 MB/s | 70 kB 00:00 2023-06-23T01:07:57.946 INFO:teuthology.orchestra.run.smithi157.stdout:(70/120): python3-zc-lockfile-2.0-2.el8.noarch. 306 kB/s | 23 kB 00:00 2023-06-23T01:07:57.963 INFO:teuthology.orchestra.run.smithi157.stdout:(71/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 555 kB/s | 19 kB 00:00 2023-06-23T01:07:57.973 INFO:teuthology.orchestra.run.smithi157.stdout:(72/120): python3-oauthlib-2.1.0-1.el8.noarch.r 10 MB/s | 155 kB 00:00 2023-06-23T01:07:57.984 INFO:teuthology.orchestra.run.smithi157.stdout:(73/120): python3-jwt-1.6.1-2.el8.noarch.rpm 1.2 MB/s | 43 kB 00:00 2023-06-23T01:07:57.991 INFO:teuthology.orchestra.run.smithi157.stdout:(74/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 12 MB/s | 220 kB 00:00 2023-06-23T01:07:57.997 INFO:teuthology.orchestra.run.smithi157.stdout:(75/120): thrift-0.13.0-2.el8.x86_64.rpm 19 MB/s | 1.7 MB 00:00 2023-06-23T01:07:58.000 INFO:teuthology.orchestra.run.smithi157.stdout:(76/120): keyutils-libs-devel-1.5.10-6.el8.x86_ 2.5 MB/s | 48 kB 00:00 2023-06-23T01:07:58.016 INFO:teuthology.orchestra.run.smithi157.stdout:(77/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 9.4 MB/s | 229 kB 00:00 2023-06-23T01:07:58.028 INFO:teuthology.orchestra.run.smithi157.stdout:(78/120): python3-requests-oauthlib-1.0.0-1.el8 1.4 MB/s | 43 kB 00:00 2023-06-23T01:07:58.033 INFO:teuthology.orchestra.run.smithi157.stdout:(79/120): libverto-devel-0.3.0-5.el8.x86_64.rpm 1.0 MB/s | 18 kB 00:00 2023-06-23T01:07:58.047 INFO:teuthology.orchestra.run.smithi157.stdout:(80/120): openssl-devel-1.1.1g-15.el8_3.x86_64. 50 MB/s | 2.3 MB 00:00 2023-06-23T01:07:58.049 INFO:teuthology.orchestra.run.smithi157.stdout:(81/120): fuse-2.9.7-12.el8.x86_64.rpm 5.0 MB/s | 83 kB 00:00 2023-06-23T01:07:58.055 INFO:teuthology.orchestra.run.smithi157.stdout:(82/120): pcre2-devel-10.32-2.el8.x86_64.rpm 21 MB/s | 605 kB 00:00 2023-06-23T01:07:58.063 INFO:teuthology.orchestra.run.smithi157.stdout:(83/120): elfutils-libelf-devel-0.182-3.el8.x86 3.5 MB/s | 59 kB 00:00 2023-06-23T01:07:58.067 INFO:teuthology.orchestra.run.smithi157.stdout:(84/120): libquadmath-8.4.1-1.el8.x86_64.rpm 9.2 MB/s | 169 kB 00:00 2023-06-23T01:07:58.081 INFO:teuthology.orchestra.run.smithi157.stdout:(85/120): libcom_err-devel-1.45.6-1.el8.x86_64. 1.4 MB/s | 38 kB 00:00 2023-06-23T01:07:58.082 INFO:teuthology.orchestra.run.smithi157.stdout:(86/120): libsepol-devel-2.9-2.el8.x86_64.rpm 4.5 MB/s | 86 kB 00:00 2023-06-23T01:07:58.093 INFO:teuthology.orchestra.run.smithi157.stdout:(87/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 2.2 MB/s | 58 kB 00:00 2023-06-23T01:07:58.098 INFO:teuthology.orchestra.run.smithi157.stdout:(88/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 2.8 MB/s | 47 kB 00:00 2023-06-23T01:07:58.106 INFO:teuthology.orchestra.run.smithi157.stdout:(89/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.rp 23 MB/s | 559 kB 00:00 2023-06-23T01:07:58.116 INFO:teuthology.orchestra.run.smithi157.stdout:(90/120): libgfortran-8.4.1-1.el8.x86_64.rpm 28 MB/s | 639 kB 00:00 2023-06-23T01:07:58.117 INFO:teuthology.orchestra.run.smithi157.stdout:(91/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rpm 3.7 MB/s | 70 kB 00:00 2023-06-23T01:07:58.126 INFO:teuthology.orchestra.run.smithi157.stdout:(92/120): libselinux-devel-2.9-5.el8.x86_64.rpm 10 MB/s | 200 kB 00:00 2023-06-23T01:07:58.135 INFO:teuthology.orchestra.run.smithi157.stdout:(93/120): libuuid-devel-2.32.1-27.el8.x86_64.rp 5.1 MB/s | 97 kB 00:00 2023-06-23T01:07:58.143 INFO:teuthology.orchestra.run.smithi157.stdout:(94/120): python3-prettytable-0.7.2-14.el8.noar 2.7 MB/s | 44 kB 00:00 2023-06-23T01:07:58.153 INFO:teuthology.orchestra.run.smithi157.stdout:(95/120): python3-jsonpatch-1.21-2.el8.noarch.r 1.5 MB/s | 27 kB 00:00 2023-06-23T01:07:58.159 INFO:teuthology.orchestra.run.smithi157.stdout:(96/120): jq-1.5-12.el8.x86_64.rpm 9.7 MB/s | 161 kB 00:00 2023-06-23T01:07:58.174 INFO:teuthology.orchestra.run.smithi157.stdout:(97/120): flex-2.6.1-9.el8.x86_64.rpm 15 MB/s | 320 kB 00:00 2023-06-23T01:07:58.183 INFO:teuthology.orchestra.run.smithi157.stdout:(98/120): python3-werkzeug-0.12.2-4.el8.noarch. 19 MB/s | 457 kB 00:00 2023-06-23T01:07:58.202 INFO:teuthology.orchestra.run.smithi157.stdout:(99/120): python3-markupsafe-0.23-19.el8.x86_64 2.1 MB/s | 39 kB 00:00 2023-06-23T01:07:58.230 INFO:teuthology.orchestra.run.smithi157.stdout:(100/120): bison-3.0.4-10.el8.x86_64.rpm 24 MB/s | 688 kB 00:00 2023-06-23T01:07:58.251 INFO:teuthology.orchestra.run.smithi157.stdout:(101/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 12 MB/s | 259 kB 00:00 2023-06-23T01:07:58.269 INFO:teuthology.orchestra.run.smithi157.stdout:(102/120): python3-babel-2.5.1-5.el8.noarch.rpm 51 MB/s | 4.8 MB 00:00 2023-06-23T01:07:58.273 INFO:teuthology.orchestra.run.smithi157.stdout:(103/120): python3-numpy-f2py-1.14.3-9.el8.x86_ 10 MB/s | 225 kB 00:00 2023-06-23T01:07:58.288 INFO:teuthology.orchestra.run.smithi157.stdout:(104/120): python3-pyasn1-0.3.7-6.el8.noarch.rp 6.4 MB/s | 126 kB 00:00 2023-06-23T01:07:58.309 INFO:teuthology.orchestra.run.smithi157.stdout:(105/120): oniguruma-6.8.2-2.el8.x86_64.rpm 8.9 MB/s | 187 kB 00:00 2023-06-23T01:07:58.420 INFO:teuthology.orchestra.run.smithi157.stdout:(106/120): python3-scipy-1.0.0-20.module+el8.1. 48 MB/s | 15 MB 00:00 2023-06-23T01:07:58.428 INFO:teuthology.orchestra.run.smithi157.stdout:(107/120): python3-jinja2-2.10.1-2.el8_0.noarch 4.4 MB/s | 538 kB 00:00 2023-06-23T01:07:58.446 INFO:teuthology.orchestra.run.smithi157.stdout:(108/120): python3-numpy-1.14.3-9.el8.x86_64.rp 22 MB/s | 3.7 MB 00:00 2023-06-23T01:07:58.449 INFO:teuthology.orchestra.run.smithi157.stdout:(109/120): python3-mako-1.0.6-13.el8.noarch.rpm 7.6 MB/s | 157 kB 00:00 2023-06-23T01:07:58.451 INFO:teuthology.orchestra.run.smithi157.stdout:(110/120): socat-1.7.3.3-2.el8.x86_64.rpm 9.7 MB/s | 302 kB 00:00 2023-06-23T01:07:58.464 INFO:teuthology.orchestra.run.smithi157.stdout:(111/120): python3-pyasn1-modules-0.3.7-6.el8.n 6.0 MB/s | 110 kB 00:00 2023-06-23T01:07:58.468 INFO:teuthology.orchestra.run.smithi157.stdout:(112/120): python3-jsonpointer-1.10-11.el8.noar 1.1 MB/s | 20 kB 00:00 2023-06-23T01:07:58.470 INFO:teuthology.orchestra.run.smithi157.stdout:(113/120): librdkafka-0.11.4-1.el8.x86_64.rpm 16 MB/s | 353 kB 00:00 2023-06-23T01:07:58.484 INFO:teuthology.orchestra.run.smithi120.stdout:(10/120): ceph-osd-17.2.6-719.g42ddc46e.el8.x86 2.7 MB/s | 17 MB 00:06 2023-06-23T01:07:58.486 INFO:teuthology.orchestra.run.smithi157.stdout:(114/120): python3-pytz-2017.2-9.el8.noarch.rpm 3.0 MB/s | 54 kB 00:00 2023-06-23T01:07:58.493 INFO:teuthology.orchestra.run.smithi157.stdout:(115/120): libtool-2.4.6-25.el8.x86_64.rpm 24 MB/s | 709 kB 00:00 2023-06-23T01:07:58.525 INFO:teuthology.orchestra.run.smithi157.stdout:(116/120): automake-1.16.1-7.el8.noarch.rpm 22 MB/s | 713 kB 00:00 2023-06-23T01:07:58.567 INFO:teuthology.orchestra.run.smithi157.stdout:(117/120): openblas-threads-0.3.12-1.el8.x86_64 49 MB/s | 4.7 MB 00:00 2023-06-23T01:07:58.569 INFO:teuthology.orchestra.run.smithi120.stdout:(11/120): cephfs-mirror-17.2.6-719.g42ddc46e.el 2.9 MB/s | 255 kB 00:00 2023-06-23T01:07:58.582 INFO:teuthology.orchestra.run.smithi157.stdout:(118/120): openblas-0.3.12-1.el8.x86_64.rpm 48 MB/s | 4.6 MB 00:00 2023-06-23T01:07:58.611 INFO:teuthology.orchestra.run.smithi120.stdout:(12/120): libcephfs-devel-17.2.6-719.g42ddc46e. 599 kB/s | 25 kB 00:00 2023-06-23T01:07:58.844 INFO:teuthology.orchestra.run.smithi120.stdout:(13/120): libcephfs2-17.2.6-719.g42ddc46e.el8.x 3.3 MB/s | 794 kB 00:00 2023-06-23T01:07:58.893 INFO:teuthology.orchestra.run.smithi120.stdout:(14/120): libcephsqlite-17.2.6-719.g42ddc46e.el 3.5 MB/s | 174 kB 00:00 2023-06-23T01:07:58.968 INFO:teuthology.orchestra.run.smithi120.stdout:(15/120): ceph-radosgw-17.2.6-719.g42ddc46e.el8 3.3 MB/s | 17 MB 00:05 2023-06-23T01:07:58.971 INFO:teuthology.orchestra.run.smithi120.stdout:(16/120): librados-devel-17.2.6-719.g42ddc46e.e 1.6 MB/s | 130 kB 00:00 2023-06-23T01:07:59.135 INFO:teuthology.orchestra.run.smithi120.stdout:(17/120): libradosstriper1-17.2.6-719.g42ddc46e 3.0 MB/s | 506 kB 00:00 2023-06-23T01:07:59.244 INFO:teuthology.orchestra.run.smithi120.stdout:(18/120): python3-ceph-argparse-17.2.6-719.g42d 409 kB/s | 45 kB 00:00 2023-06-23T01:07:59.304 INFO:teuthology.orchestra.run.smithi120.stdout:(19/120): python3-ceph-common-17.2.6-719.g42ddc 1.9 MB/s | 115 kB 00:00 2023-06-23T01:07:59.371 INFO:teuthology.orchestra.run.smithi120.stdout:(20/120): python3-cephfs-17.2.6-719.g42ddc46e.e 2.8 MB/s | 190 kB 00:00 2023-06-23T01:07:59.507 INFO:teuthology.orchestra.run.smithi120.stdout:(21/120): python3-rados-17.2.6-719.g42ddc46e.el 2.7 MB/s | 371 kB 00:00 2023-06-23T01:07:59.643 INFO:teuthology.orchestra.run.smithi120.stdout:(22/120): python3-rbd-17.2.6-719.g42ddc46e.el8. 2.6 MB/s | 357 kB 00:00 2023-06-23T01:07:59.709 INFO:teuthology.orchestra.run.smithi120.stdout:(23/120): python3-rgw-17.2.6-719.g42ddc46e.el8. 1.6 MB/s | 107 kB 00:00 2023-06-23T01:07:59.776 INFO:teuthology.orchestra.run.smithi120.stdout:(24/120): rbd-fuse-17.2.6-719.g42ddc46e.el8.x86 1.3 MB/s | 91 kB 00:00 2023-06-23T01:07:59.883 INFO:teuthology.orchestra.run.smithi157.stdout:(119/120): librados2-17.2.6-719.g42ddc46e.el8.x 2.6 MB/s | 3.5 MB 00:01 2023-06-23T01:07:59.937 INFO:teuthology.orchestra.run.smithi157.stdout:(120/120): librbd1-17.2.6-719.g42ddc46e.el8.x86 2.6 MB/s | 3.6 MB 00:01 2023-06-23T01:07:59.940 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2023-06-23T01:07:59.941 INFO:teuthology.orchestra.run.smithi157.stdout:Total 15 MB/s | 201 MB 00:13 2023-06-23T01:07:59.988 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2023-06-23T01:08:00.101 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2023-06-23T01:08:00.102 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2023-06-23T01:08:00.198 INFO:teuthology.orchestra.run.smithi120.stdout:(25/120): ceph-test-17.2.6-719.g42ddc46e.el8.x8 8.4 MB/s | 52 MB 00:06 2023-06-23T01:08:00.232 INFO:teuthology.orchestra.run.smithi120.stdout:(26/120): rbd-nbd-17.2.6-719.g42ddc46e.el8.x86_ 5.0 MB/s | 173 kB 00:00 2023-06-23T01:08:00.272 INFO:teuthology.orchestra.run.smithi120.stdout:(27/120): ceph-grafana-dashboards-17.2.6-719.g4 580 kB/s | 23 kB 00:00 2023-06-23T01:08:00.311 INFO:teuthology.orchestra.run.smithi120.stdout:(28/120): ceph-mgr-cephadm-17.2.6-719.g42ddc46e 3.0 MB/s | 121 kB 00:00 2023-06-23T01:08:00.488 INFO:teuthology.orchestra.run.smithi134.stdout:(10/120): ceph-radosgw-17.2.6-719.g42ddc46e.el8 1.8 MB/s | 17 MB 00:09 2023-06-23T01:08:00.499 INFO:teuthology.orchestra.run.smithi120.stdout:(29/120): ceph-mgr-dashboard-17.2.6-719.g42ddc4 9.1 MB/s | 1.7 MB 00:00 2023-06-23T01:08:00.517 INFO:teuthology.orchestra.run.smithi120.stdout:(30/120): librgw2-17.2.6-719.g42ddc46e.el8.x86_ 3.6 MB/s | 5.5 MB 00:01 2023-06-23T01:08:00.569 INFO:teuthology.orchestra.run.smithi120.stdout:(31/120): rbd-mirror-17.2.6-719.g42ddc46e.el8.x 4.4 MB/s | 3.5 MB 00:00 2023-06-23T01:08:00.602 INFO:teuthology.orchestra.run.smithi134.stdout:(11/120): cephfs-mirror-17.2.6-719.g42ddc46e.el 2.1 MB/s | 255 kB 00:00 2023-06-23T01:08:00.605 INFO:teuthology.orchestra.run.smithi120.stdout:(32/120): ceph-mgr-modules-core-17.2.6-719.g42d 2.6 MB/s | 238 kB 00:00 2023-06-23T01:08:00.606 INFO:teuthology.orchestra.run.smithi120.stdout:(33/120): ceph-mgr-rook-17.2.6-719.g42ddc46e.el 1.3 MB/s | 47 kB 00:00 2023-06-23T01:08:00.638 INFO:teuthology.orchestra.run.smithi120.stdout:(34/120): ceph-prometheus-alerts-17.2.6-719.g42 449 kB/s | 14 kB 00:00 2023-06-23T01:08:00.652 INFO:teuthology.orchestra.run.smithi134.stdout:(12/120): libcephfs-devel-17.2.6-719.g42ddc46e. 490 kB/s | 25 kB 00:00 2023-06-23T01:08:00.685 INFO:teuthology.orchestra.run.smithi120.stdout:(35/120): cephadm-17.2.6-719.g42ddc46e.el8.noar 1.7 MB/s | 80 kB 00:00 2023-06-23T01:08:00.686 INFO:teuthology.orchestra.run.smithi120.stdout:(36/120): ceph-volume-17.2.6-719.g42ddc46e.el8. 3.2 MB/s | 264 kB 00:00 2023-06-23T01:08:00.703 INFO:teuthology.orchestra.run.smithi120.stdout:[MIRROR] gperftools-libs-2.7-9.el8.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/g/gperftools-libs-2.7-9.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T01:08:00.706 INFO:teuthology.orchestra.run.smithi120.stdout:[MIRROR] gperftools-libs-2.7-9.el8.x86_64.rpm: Status code: 404 for http://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/g/gperftools-libs-2.7-9.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T01:08:00.724 INFO:teuthology.orchestra.run.smithi120.stdout:(37/120): cephfs-top-17.2.6-719.g42ddc46e.el8.n 455 kB/s | 18 kB 00:00 2023-06-23T01:08:00.725 INFO:teuthology.orchestra.run.smithi120.stdout:[MIRROR] liboath-2.6.2-3.el8.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/l/liboath-2.6.2-3.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T01:08:00.726 INFO:teuthology.orchestra.run.smithi120.stdout:[MIRROR] liboath-2.6.2-3.el8.x86_64.rpm: Status code: 404 for http://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/l/liboath-2.6.2-3.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T01:08:00.795 INFO:teuthology.orchestra.run.smithi120.stdout:(38/120): liboath-2.6.2-3.el8.x86_64.rpm 845 kB/s | 59 kB 00:00 2023-06-23T01:08:00.795 INFO:teuthology.orchestra.run.smithi120.stdout:[MIRROR] libunwind-1.3.1-3.el8.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/l/libunwind-1.3.1-3.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T01:08:00.796 INFO:teuthology.orchestra.run.smithi120.stdout:[MIRROR] libunwind-1.3.1-3.el8.x86_64.rpm: Status code: 404 for http://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/l/libunwind-1.3.1-3.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T01:08:00.815 INFO:teuthology.orchestra.run.smithi120.stdout:(39/120): gperftools-libs-2.7-9.el8.x86_64.rpm 2.3 MB/s | 306 kB 00:00 2023-06-23T01:08:00.816 INFO:teuthology.orchestra.run.smithi120.stdout:[MIRROR] python3-asyncssh-2.7.0-2.el8.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/p/python3-asyncssh-2.7.0-2.el8.noarch.rpm (IP: 8.43.85.72) 2023-06-23T01:08:00.817 INFO:teuthology.orchestra.run.smithi120.stdout:[MIRROR] python3-asyncssh-2.7.0-2.el8.noarch.rpm: Status code: 404 for http://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/p/python3-asyncssh-2.7.0-2.el8.noarch.rpm (IP: 8.43.85.72) 2023-06-23T01:08:00.843 INFO:teuthology.orchestra.run.smithi120.stdout:(40/120): python3-asyncssh-2.7.0-2.el8.noarch.r 17 MB/s | 476 kB 00:00 2023-06-23T01:08:00.857 INFO:teuthology.orchestra.run.smithi120.stdout:(41/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 3.1 MB/s | 44 kB 00:00 2023-06-23T01:08:00.872 INFO:teuthology.orchestra.run.smithi120.stdout:(42/120): python3-beautifulsoup4-4.6.3-2.el8.1. 13 MB/s | 185 kB 00:00 2023-06-23T01:08:00.879 INFO:teuthology.orchestra.run.smithi120.stdout:(43/120): libunwind-1.3.1-3.el8.x86_64.rpm 890 kB/s | 75 kB 00:00 2023-06-23T01:08:00.887 INFO:teuthology.orchestra.run.smithi120.stdout:(44/120): python3-cachetools-3.1.1-4.el8.noarch 2.1 MB/s | 33 kB 00:00 2023-06-23T01:08:00.891 INFO:teuthology.orchestra.run.smithi120.stdout:(45/120): python3-certifi-2018.10.15-7.el8.noar 1.3 MB/s | 16 kB 00:00 2023-06-23T01:08:00.902 INFO:teuthology.orchestra.run.smithi120.stdout:(46/120): python3-cheroot-8.5.2-1.el8.noarch.rp 12 MB/s | 173 kB 00:00 2023-06-23T01:08:00.908 INFO:teuthology.orchestra.run.smithi134.stdout:(13/120): ceph-common-17.2.6-719.g42ddc46e.el8. 1.7 MB/s | 23 MB 00:13 2023-06-23T01:08:00.915 INFO:teuthology.orchestra.run.smithi120.stdout:(47/120): python3-google-auth-1.1.1-10.el8.noar 6.4 MB/s | 81 kB 00:00 2023-06-23T01:08:00.929 INFO:teuthology.orchestra.run.smithi120.stdout:(48/120): python3-jaraco-6.2-6.el8.noarch.rpm 785 kB/s | 11 kB 00:00 2023-06-23T01:08:00.939 INFO:teuthology.orchestra.run.smithi120.stdout:(49/120): python3-cherrypy-18.4.0-1.el8.noarch. 7.8 MB/s | 384 kB 00:00 2023-06-23T01:08:00.941 INFO:teuthology.orchestra.run.smithi120.stdout:(50/120): python3-jaraco-functools-2.0-4.el8.no 1.5 MB/s | 18 kB 00:00 2023-06-23T01:08:00.957 INFO:teuthology.orchestra.run.smithi120.stdout:(51/120): python3-logutils-0.3.5-11.el8.noarch. 3.6 MB/s | 49 kB 00:00 2023-06-23T01:08:00.967 INFO:teuthology.orchestra.run.smithi120.stdout:(52/120): python3-more-itertools-7.2.0-3.el8.no 4.7 MB/s | 59 kB 00:00 2023-06-23T01:08:00.968 INFO:teuthology.orchestra.run.smithi134.stdout:(14/120): libcephfs2-17.2.6-719.g42ddc46e.el8.x 2.5 MB/s | 794 kB 00:00 2023-06-23T01:08:00.970 INFO:teuthology.orchestra.run.smithi134.stdout:(15/120): libcephsqlite-17.2.6-719.g42ddc46e.el 2.6 MB/s | 174 kB 00:00 2023-06-23T01:08:00.979 INFO:teuthology.orchestra.run.smithi120.stdout:(53/120): python3-natsort-7.1.1-2.el8.noarch.rp 4.9 MB/s | 60 kB 00:00 2023-06-23T01:08:01.005 INFO:teuthology.orchestra.run.smithi120.stdout:(54/120): ceph-mgr-diskprediction-local-17.2.6- 15 MB/s | 7.4 MB 00:00 2023-06-23T01:08:01.008 INFO:teuthology.orchestra.run.smithi120.stdout:(55/120): python3-pecan-1.3.2-9.el8.noarch.rpm 9.7 MB/s | 283 kB 00:00 2023-06-23T01:08:01.009 INFO:teuthology.orchestra.run.smithi134.stdout:(16/120): librados-devel-17.2.6-719.g42ddc46e.e 3.2 MB/s | 130 kB 00:00 2023-06-23T01:08:01.024 INFO:teuthology.orchestra.run.smithi120.stdout:(56/120): python3-repoze-lru-0.7-6.el8.noarch.r 2.1 MB/s | 34 kB 00:00 2023-06-23T01:08:01.040 INFO:teuthology.orchestra.run.smithi120.stdout:(57/120): python3-routes-2.4.1-12.el8.noarch.rp 13 MB/s | 196 kB 00:00 2023-06-23T01:08:01.053 INFO:teuthology.orchestra.run.smithi120.stdout:(58/120): python3-rsa-4.8-1.el8.noarch.rpm 4.9 MB/s | 61 kB 00:00 2023-06-23T01:08:01.056 INFO:teuthology.orchestra.run.smithi134.stdout:(17/120): libradosstriper1-17.2.6-719.g42ddc46e 5.7 MB/s | 506 kB 00:00 2023-06-23T01:08:01.058 INFO:teuthology.orchestra.run.smithi120.stdout:(59/120): python3-kubernetes-11.0.0-6.el8.noarc 9.9 MB/s | 1.2 MB 00:00 2023-06-23T01:08:01.062 INFO:teuthology.orchestra.run.smithi120.stdout:(60/120): python3-portend-2.6-1.el8.noarch.rpm 272 kB/s | 16 kB 00:00 2023-06-23T01:08:01.065 INFO:teuthology.orchestra.run.smithi120.stdout:(61/120): python3-simplegeneric-0.8.1-17.el8.no 1.5 MB/s | 19 kB 00:00 2023-06-23T01:08:01.069 INFO:teuthology.orchestra.run.smithi120.stdout:(62/120): python3-singledispatch-3.4.0.3-18.el8 2.0 MB/s | 24 kB 00:00 2023-06-23T01:08:01.075 INFO:teuthology.orchestra.run.smithi120.stdout:(63/120): python3-tempora-1.14.1-5.el8.noarch.r 2.4 MB/s | 29 kB 00:00 2023-06-23T01:08:01.077 INFO:teuthology.orchestra.run.smithi120.stdout:(64/120): python3-trustme-0.6.0-4.el8.noarch.rp 2.2 MB/s | 27 kB 00:00 2023-06-23T01:08:01.090 INFO:teuthology.orchestra.run.smithi120.stdout:(65/120): python3-websocket-client-0.56.0-5.el8 4.8 MB/s | 61 kB 00:00 2023-06-23T01:08:01.094 INFO:teuthology.orchestra.run.smithi134.stdout:(18/120): python3-ceph-argparse-17.2.6-719.g42d 1.2 MB/s | 45 kB 00:00 2023-06-23T01:08:01.095 INFO:teuthology.orchestra.run.smithi120.stdout:(66/120): python3-waitress-1.4.3-1.el8.noarch.r 10 MB/s | 255 kB 00:00 2023-06-23T01:08:01.103 INFO:teuthology.orchestra.run.smithi120.stdout:(67/120): python3-webtest-2.0.33-1.el8.noarch.r 6.7 MB/s | 85 kB 00:00 2023-06-23T01:08:01.108 INFO:teuthology.orchestra.run.smithi120.stdout:(68/120): python3-zc-lockfile-2.0-2.el8.noarch. 1.9 MB/s | 23 kB 00:00 2023-06-23T01:08:01.113 INFO:teuthology.orchestra.run.smithi120.stdout:(69/120): python3-webob-1.8.5-1.el8.1.noarch.rp 7.0 MB/s | 251 kB 00:00 2023-06-23T01:08:01.113 INFO:teuthology.orchestra.run.smithi120.stdout:(70/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 15 MB/s | 70 kB 00:00 2023-06-23T01:08:01.132 INFO:teuthology.orchestra.run.smithi134.stdout:(19/120): python3-ceph-common-17.2.6-719.g42ddc 3.0 MB/s | 115 kB 00:00 2023-06-23T01:08:01.154 INFO:teuthology.orchestra.run.smithi120.stdout:(71/120): thrift-0.13.0-2.el8.x86_64.rpm 34 MB/s | 1.7 MB 00:00 2023-06-23T01:08:01.162 INFO:teuthology.orchestra.run.smithi120.stdout:(72/120): python3-prettytable-0.7.2-14.el8.noar 900 kB/s | 44 kB 00:00 2023-06-23T01:08:01.170 INFO:teuthology.orchestra.run.smithi134.stdout:(20/120): python3-cephfs-17.2.6-719.g42ddc46e.e 4.9 MB/s | 190 kB 00:00 2023-06-23T01:08:01.188 INFO:teuthology.orchestra.run.smithi120.stdout:(73/120): jq-1.5-12.el8.x86_64.rpm 6.0 MB/s | 161 kB 00:00 2023-06-23T01:08:01.195 INFO:teuthology.orchestra.run.smithi120.stdout:(74/120): python3-jsonpatch-1.21-2.el8.noarch.r 653 kB/s | 27 kB 00:00 2023-06-23T01:08:01.214 INFO:teuthology.orchestra.run.smithi120.stdout:(75/120): flex-2.6.1-9.el8.x86_64.rpm 12 MB/s | 320 kB 00:00 2023-06-23T01:08:01.224 INFO:teuthology.orchestra.run.smithi120.stdout:(76/120): python3-werkzeug-0.12.2-4.el8.noarch. 16 MB/s | 457 kB 00:00 2023-06-23T01:08:01.242 INFO:teuthology.orchestra.run.smithi120.stdout:(77/120): python3-markupsafe-0.23-19.el8.x86_64 2.1 MB/s | 39 kB 00:00 2023-06-23T01:08:01.244 INFO:teuthology.orchestra.run.smithi134.stdout:(21/120): python3-rados-17.2.6-719.g42ddc46e.el 4.9 MB/s | 371 kB 00:00 2023-06-23T01:08:01.280 INFO:teuthology.orchestra.run.smithi120.stdout:(78/120): bison-3.0.4-10.el8.x86_64.rpm 19 MB/s | 688 kB 00:00 2023-06-23T01:08:01.303 INFO:teuthology.orchestra.run.smithi120.stdout:(79/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 10 MB/s | 259 kB 00:00 2023-06-23T01:08:01.317 INFO:teuthology.orchestra.run.smithi134.stdout:(22/120): python3-rbd-17.2.6-719.g42ddc46e.el8. 4.8 MB/s | 357 kB 00:00 2023-06-23T01:08:01.326 INFO:teuthology.orchestra.run.smithi120.stdout:(80/120): python3-numpy-f2py-1.14.3-9.el8.x86_6 9.7 MB/s | 225 kB 00:00 2023-06-23T01:08:01.347 INFO:teuthology.orchestra.run.smithi120.stdout:(81/120): python3-babel-2.5.1-5.el8.noarch.rpm 36 MB/s | 4.8 MB 00:00 2023-06-23T01:08:01.350 INFO:teuthology.orchestra.run.smithi120.stdout:(82/120): python3-pyasn1-0.3.7-6.el8.noarch.rpm 5.1 MB/s | 126 kB 00:00 2023-06-23T01:08:01.354 INFO:teuthology.orchestra.run.smithi134.stdout:(23/120): python3-rgw-17.2.6-719.g42ddc46e.el8. 2.8 MB/s | 107 kB 00:00 2023-06-23T01:08:01.372 INFO:teuthology.orchestra.run.smithi120.stdout:(83/120): oniguruma-6.8.2-2.el8.x86_64.rpm 8.5 MB/s | 187 kB 00:00 2023-06-23T01:08:01.391 INFO:teuthology.orchestra.run.smithi134.stdout:(24/120): rbd-fuse-17.2.6-719.g42ddc46e.el8.x86 2.4 MB/s | 91 kB 00:00 2023-06-23T01:08:01.403 INFO:teuthology.orchestra.run.smithi120.stdout:(84/120): python3-jinja2-2.10.1-2.el8_0.noarch. 17 MB/s | 538 kB 00:00 2023-06-23T01:08:01.429 INFO:teuthology.orchestra.run.smithi120.stdout:(85/120): socat-1.7.3.3-2.el8.x86_64.rpm 12 MB/s | 302 kB 00:00 2023-06-23T01:08:01.455 INFO:teuthology.orchestra.run.smithi120.stdout:(86/120): python3-numpy-1.14.3-9.el8.x86_64.rpm 35 MB/s | 3.7 MB 00:00 2023-06-23T01:08:01.459 INFO:teuthology.orchestra.run.smithi120.stdout:(87/120): python3-mako-1.0.6-13.el8.noarch.rpm 5.2 MB/s | 157 kB 00:00 2023-06-23T01:08:01.507 INFO:teuthology.orchestra.run.smithi120.stdout:(88/120): python3-scipy-1.0.0-20.module+el8.1.0 37 MB/s | 15 MB 00:00 2023-06-23T01:08:01.510 INFO:teuthology.orchestra.run.smithi120.stdout:(89/120): python3-pyasn1-modules-0.3.7-6.el8.no 2.0 MB/s | 110 kB 00:00 2023-06-23T01:08:01.514 INFO:teuthology.orchestra.run.smithi120.stdout:(90/120): librdkafka-0.11.4-1.el8.x86_64.rpm 6.3 MB/s | 353 kB 00:00 2023-06-23T01:08:01.526 INFO:teuthology.orchestra.run.smithi120.stdout:(91/120): python3-jsonpointer-1.10-11.el8.noarc 1.1 MB/s | 20 kB 00:00 2023-06-23T01:08:01.537 INFO:teuthology.orchestra.run.smithi120.stdout:(92/120): python3-pytz-2017.2-9.el8.noarch.rpm 2.3 MB/s | 54 kB 00:00 2023-06-23T01:08:01.541 INFO:teuthology.orchestra.run.smithi120.stdout:(93/120): libtool-2.4.6-25.el8.x86_64.rpm 23 MB/s | 709 kB 00:00 2023-06-23T01:08:01.578 INFO:teuthology.orchestra.run.smithi120.stdout:(94/120): automake-1.16.1-7.el8.noarch.rpm 19 MB/s | 713 kB 00:00 2023-06-23T01:08:01.593 INFO:teuthology.orchestra.run.smithi120.stdout:(95/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 1.2 MB/s | 19 kB 00:00 2023-06-23T01:08:01.608 INFO:teuthology.orchestra.run.smithi120.stdout:(96/120): python3-jwt-1.6.1-2.el8.noarch.rpm 3.0 MB/s | 43 kB 00:00 2023-06-23T01:08:01.627 INFO:teuthology.orchestra.run.smithi120.stdout:(97/120): python3-oauthlib-2.1.0-1.el8.noarch.r 8.1 MB/s | 155 kB 00:00 2023-06-23T01:08:01.659 INFO:teuthology.orchestra.run.smithi120.stdout:(98/120): openblas-threads-0.3.12-1.el8.x86_64. 35 MB/s | 4.7 MB 00:00 2023-06-23T01:08:01.661 INFO:teuthology.orchestra.run.smithi120.stdout:(99/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 6.2 MB/s | 220 kB 00:00 2023-06-23T01:08:01.681 INFO:teuthology.orchestra.run.smithi120.stdout:(100/120): openblas-0.3.12-1.el8.x86_64.rpm 32 MB/s | 4.6 MB 00:00 2023-06-23T01:08:01.682 INFO:teuthology.orchestra.run.smithi120.stdout:(101/120): keyutils-libs-devel-1.5.10-6.el8.x86 2.1 MB/s | 48 kB 00:00 2023-06-23T01:08:01.685 INFO:teuthology.orchestra.run.smithi120.stdout:(102/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 9.6 MB/s | 229 kB 00:00 2023-06-23T01:08:01.697 INFO:teuthology.orchestra.run.smithi120.stdout:(103/120): python3-requests-oauthlib-1.0.0-1.el 2.5 MB/s | 43 kB 00:00 2023-06-23T01:08:01.703 INFO:teuthology.orchestra.run.smithi120.stdout:(104/120): libverto-devel-0.3.0-5.el8.x86_64.rp 998 kB/s | 18 kB 00:00 2023-06-23T01:08:01.722 INFO:teuthology.orchestra.run.smithi120.stdout:(105/120): fuse-2.9.7-12.el8.x86_64.rpm 4.4 MB/s | 83 kB 00:00 2023-06-23T01:08:01.728 INFO:teuthology.orchestra.run.smithi120.stdout:(106/120): pcre2-devel-10.32-2.el8.x86_64.rpm 19 MB/s | 605 kB 00:00 2023-06-23T01:08:01.745 INFO:teuthology.orchestra.run.smithi120.stdout:(107/120): openssl-devel-1.1.1g-15.el8_3.x86_64 37 MB/s | 2.3 MB 00:00 2023-06-23T01:08:01.747 INFO:teuthology.orchestra.run.smithi120.stdout:(108/120): elfutils-libelf-devel-0.182-3.el8.x8 2.3 MB/s | 59 kB 00:00 2023-06-23T01:08:01.749 INFO:teuthology.orchestra.run.smithi120.stdout:(109/120): libquadmath-8.4.1-1.el8.x86_64.rpm 8.1 MB/s | 169 kB 00:00 2023-06-23T01:08:01.763 INFO:teuthology.orchestra.run.smithi120.stdout:(110/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 4.0 MB/s | 58 kB 00:00 2023-06-23T01:08:01.767 INFO:teuthology.orchestra.run.smithi120.stdout:(111/120): libcom_err-devel-1.45.6-1.el8.x86_64 1.7 MB/s | 38 kB 00:00 2023-06-23T01:08:01.774 INFO:teuthology.orchestra.run.smithi120.stdout:(112/120): libsepol-devel-2.9-2.el8.x86_64.rpm 3.1 MB/s | 86 kB 00:00 2023-06-23T01:08:01.783 INFO:teuthology.orchestra.run.smithi120.stdout:(113/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 2.3 MB/s | 47 kB 00:00 2023-06-23T01:08:01.792 INFO:teuthology.orchestra.run.smithi120.stdout:(114/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.r 22 MB/s | 559 kB 00:00 2023-06-23T01:08:01.798 INFO:teuthology.orchestra.run.smithi120.stdout:(115/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rp 4.6 MB/s | 70 kB 00:00 2023-06-23T01:08:01.803 INFO:teuthology.orchestra.run.smithi120.stdout:(116/120): libgfortran-8.4.1-1.el8.x86_64.rpm 22 MB/s | 639 kB 00:00 2023-06-23T01:08:01.811 INFO:teuthology.orchestra.run.smithi120.stdout:(117/120): libselinux-devel-2.9-5.el8.x86_64.rp 10 MB/s | 200 kB 00:00 2023-06-23T01:08:01.814 INFO:teuthology.orchestra.run.smithi120.stdout:(118/120): libuuid-devel-2.32.1-27.el8.x86_64.r 6.1 MB/s | 97 kB 00:00 2023-06-23T01:08:01.817 INFO:teuthology.orchestra.run.smithi134.stdout:(25/120): librgw2-17.2.6-719.g42ddc46e.el8.x86_ 6.9 MB/s | 5.5 MB 00:00 2023-06-23T01:08:01.870 INFO:teuthology.orchestra.run.smithi134.stdout:(26/120): rbd-nbd-17.2.6-719.g42ddc46e.el8.x86_ 3.2 MB/s | 173 kB 00:00 2023-06-23T01:08:01.895 INFO:teuthology.orchestra.run.smithi134.stdout:(27/120): rbd-mirror-17.2.6-719.g42ddc46e.el8.x 6.9 MB/s | 3.5 MB 00:00 2023-06-23T01:08:01.921 INFO:teuthology.orchestra.run.smithi134.stdout:(28/120): ceph-grafana-dashboards-17.2.6-719.g4 458 kB/s | 23 kB 00:00 2023-06-23T01:08:01.952 INFO:teuthology.orchestra.run.smithi134.stdout:(29/120): ceph-mgr-cephadm-17.2.6-719.g42ddc46e 2.0 MB/s | 121 kB 00:00 2023-06-23T01:08:02.000 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2023-06-23T01:08:02.006 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2023-06-23T01:08:02.318 INFO:teuthology.orchestra.run.smithi134.stdout:(30/120): ceph-mgr-dashboard-17.2.6-719.g42ddc4 4.3 MB/s | 1.7 MB 00:00 2023-06-23T01:08:02.350 INFO:teuthology.orchestra.run.smithi120.stdout:(119/120): librados2-17.2.6-719.g42ddc46e.el8.x 6.3 MB/s | 3.5 MB 00:00 2023-06-23T01:08:02.367 INFO:teuthology.orchestra.run.smithi134.stdout:(31/120): ceph-mgr-modules-core-17.2.6-719.g42d 4.7 MB/s | 238 kB 00:00 2023-06-23T01:08:02.376 INFO:teuthology.orchestra.run.smithi120.stdout:(120/120): librbd1-17.2.6-719.g42ddc46e.el8.x86 6.4 MB/s | 3.6 MB 00:00 2023-06-23T01:08:02.382 INFO:teuthology.orchestra.run.smithi120.stdout:-------------------------------------------------------------------------------- 2023-06-23T01:08:02.382 INFO:teuthology.orchestra.run.smithi120.stdout:Total 15 MB/s | 201 MB 00:13 2023-06-23T01:08:02.404 INFO:teuthology.orchestra.run.smithi134.stdout:(32/120): ceph-mgr-rook-17.2.6-719.g42ddc46e.el 1.2 MB/s | 47 kB 00:00 2023-06-23T01:08:02.440 INFO:teuthology.orchestra.run.smithi134.stdout:(33/120): ceph-prometheus-alerts-17.2.6-719.g42 404 kB/s | 14 kB 00:00 2023-06-23T01:08:02.450 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction check 2023-06-23T01:08:02.485 INFO:teuthology.orchestra.run.smithi134.stdout:(34/120): ceph-volume-17.2.6-719.g42ddc46e.el8. 6.0 MB/s | 264 kB 00:00 2023-06-23T01:08:02.521 INFO:teuthology.orchestra.run.smithi134.stdout:(35/120): cephadm-17.2.6-719.g42ddc46e.el8.noar 2.1 MB/s | 80 kB 00:00 2023-06-23T01:08:02.565 INFO:teuthology.orchestra.run.smithi134.stdout:(36/120): cephfs-top-17.2.6-719.g42ddc46e.el8.n 401 kB/s | 18 kB 00:00 2023-06-23T01:08:02.574 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction check succeeded. 2023-06-23T01:08:02.574 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction test 2023-06-23T01:08:02.583 INFO:teuthology.orchestra.run.smithi134.stdout:[MIRROR] gperftools-libs-2.7-9.el8.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/g/gperftools-libs-2.7-9.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T01:08:02.585 INFO:teuthology.orchestra.run.smithi134.stdout:[MIRROR] gperftools-libs-2.7-9.el8.x86_64.rpm: Status code: 404 for http://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/g/gperftools-libs-2.7-9.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T01:08:02.802 INFO:teuthology.orchestra.run.smithi134.stdout:(37/120): gperftools-libs-2.7-9.el8.x86_64.rpm 1.3 MB/s | 306 kB 00:00 2023-06-23T01:08:02.803 INFO:teuthology.orchestra.run.smithi134.stdout:[MIRROR] liboath-2.6.2-3.el8.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/l/liboath-2.6.2-3.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T01:08:02.804 INFO:teuthology.orchestra.run.smithi134.stdout:[MIRROR] liboath-2.6.2-3.el8.x86_64.rpm: Status code: 404 for http://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/l/liboath-2.6.2-3.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T01:08:02.841 INFO:teuthology.orchestra.run.smithi134.stdout:(38/120): liboath-2.6.2-3.el8.x86_64.rpm 1.5 MB/s | 59 kB 00:00 2023-06-23T01:08:02.842 INFO:teuthology.orchestra.run.smithi134.stdout:[MIRROR] libunwind-1.3.1-3.el8.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/l/libunwind-1.3.1-3.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T01:08:02.843 INFO:teuthology.orchestra.run.smithi134.stdout:[MIRROR] libunwind-1.3.1-3.el8.x86_64.rpm: Status code: 404 for http://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/l/libunwind-1.3.1-3.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T01:08:02.877 INFO:teuthology.orchestra.run.smithi134.stdout:(39/120): libunwind-1.3.1-3.el8.x86_64.rpm 2.0 MB/s | 75 kB 00:00 2023-06-23T01:08:03.322 INFO:teuthology.orchestra.run.smithi134.stdout:(40/120): ceph-test-17.2.6-719.g42ddc46e.el8.x8 6.1 MB/s | 52 MB 00:08 2023-06-23T01:08:03.324 INFO:teuthology.orchestra.run.smithi134.stdout:(41/120): python3-asyncssh-2.7.0-2.el8.noarch.r 1.0 MB/s | 476 kB 00:00 2023-06-23T01:08:03.380 INFO:teuthology.orchestra.run.smithi134.stdout:(42/120): python3-beautifulsoup4-4.6.3-2.el8.1. 3.2 MB/s | 185 kB 00:00 2023-06-23T01:08:03.417 INFO:teuthology.orchestra.run.smithi134.stdout:(43/120): python3-cachetools-3.1.1-4.el8.noarch 902 kB/s | 33 kB 00:00 2023-06-23T01:08:03.454 INFO:teuthology.orchestra.run.smithi134.stdout:(44/120): python3-certifi-2018.10.15-7.el8.noar 447 kB/s | 16 kB 00:00 2023-06-23T01:08:03.477 INFO:teuthology.orchestra.run.smithi134.stdout:(45/120): ceph-mgr-diskprediction-local-17.2.6- 4.9 MB/s | 7.4 MB 00:01 2023-06-23T01:08:03.485 INFO:teuthology.orchestra.run.smithi134.stdout:(46/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 267 kB/s | 44 kB 00:00 2023-06-23T01:08:03.499 INFO:teuthology.orchestra.run.smithi134.stdout:(47/120): python3-cheroot-8.5.2-1.el8.noarch.rp 3.8 MB/s | 173 kB 00:00 2023-06-23T01:08:03.528 INFO:teuthology.orchestra.run.smithi134.stdout:(48/120): python3-jaraco-6.2-6.el8.noarch.rpm 381 kB/s | 11 kB 00:00 2023-06-23T01:08:03.558 INFO:teuthology.orchestra.run.smithi134.stdout:(49/120): python3-google-auth-1.1.1-10.el8.noar 1.1 MB/s | 81 kB 00:00 2023-06-23T01:08:03.559 INFO:teuthology.orchestra.run.smithi134.stdout:(50/120): python3-jaraco-functools-2.0-4.el8.no 591 kB/s | 18 kB 00:00 2023-06-23T01:08:03.604 INFO:teuthology.orchestra.run.smithi134.stdout:(51/120): python3-logutils-0.3.5-11.el8.noarch. 1.0 MB/s | 49 kB 00:00 2023-06-23T01:08:03.641 INFO:teuthology.orchestra.run.smithi134.stdout:(52/120): python3-kubernetes-11.0.0-6.el8.noarc 14 MB/s | 1.2 MB 00:00 2023-06-23T01:08:03.645 INFO:teuthology.orchestra.run.smithi134.stdout:(53/120): python3-more-itertools-7.2.0-3.el8.no 1.4 MB/s | 59 kB 00:00 2023-06-23T01:08:03.686 INFO:teuthology.orchestra.run.smithi134.stdout:(54/120): python3-cherrypy-18.4.0-1.el8.noarch. 1.8 MB/s | 384 kB 00:00 2023-06-23T01:08:03.701 INFO:teuthology.orchestra.run.smithi134.stdout:(55/120): python3-natsort-7.1.1-2.el8.noarch.rp 1.0 MB/s | 60 kB 00:00 2023-06-23T01:08:03.717 INFO:teuthology.orchestra.run.smithi134.stdout:(56/120): python3-portend-2.6-1.el8.noarch.rpm 518 kB/s | 16 kB 00:00 2023-06-23T01:08:03.731 INFO:teuthology.orchestra.run.smithi134.stdout:(57/120): python3-repoze-lru-0.7-6.el8.noarch.r 1.1 MB/s | 34 kB 00:00 2023-06-23T01:08:03.756 INFO:teuthology.orchestra.run.smithi134.stdout:(58/120): python3-routes-2.4.1-12.el8.noarch.rp 4.9 MB/s | 196 kB 00:00 2023-06-23T01:08:03.764 INFO:teuthology.orchestra.run.smithi134.stdout:(59/120): python3-rsa-4.8-1.el8.noarch.rpm 1.8 MB/s | 61 kB 00:00 2023-06-23T01:08:03.769 INFO:teuthology.orchestra.run.smithi134.stdout:(60/120): python3-pecan-1.3.2-9.el8.noarch.rpm 2.2 MB/s | 283 kB 00:00 2023-06-23T01:08:03.787 INFO:teuthology.orchestra.run.smithi134.stdout:(61/120): python3-simplegeneric-0.8.1-17.el8.no 619 kB/s | 19 kB 00:00 2023-06-23T01:08:03.794 INFO:teuthology.orchestra.run.smithi134.stdout:(62/120): python3-singledispatch-3.4.0.3-18.el8 818 kB/s | 24 kB 00:00 2023-06-23T01:08:03.799 INFO:teuthology.orchestra.run.smithi134.stdout:(63/120): python3-tempora-1.14.1-5.el8.noarch.r 985 kB/s | 29 kB 00:00 2023-06-23T01:08:03.817 INFO:teuthology.orchestra.run.smithi134.stdout:(64/120): python3-trustme-0.6.0-4.el8.noarch.rp 933 kB/s | 27 kB 00:00 2023-06-23T01:08:03.827 INFO:teuthology.orchestra.run.smithi134.stdout:(65/120): python3-waitress-1.4.3-1.el8.noarch.r 7.6 MB/s | 255 kB 00:00 2023-06-23T01:08:03.844 INFO:teuthology.orchestra.run.smithi134.stdout:(66/120): python3-webob-1.8.5-1.el8.1.noarch.rp 5.5 MB/s | 251 kB 00:00 2023-06-23T01:08:03.856 INFO:teuthology.orchestra.run.smithi134.stdout:(67/120): python3-websocket-client-0.56.0-5.el8 1.5 MB/s | 61 kB 00:00 2023-06-23T01:08:03.857 INFO:teuthology.orchestra.run.smithi134.stdout:(68/120): python3-webtest-2.0.33-1.el8.noarch.r 2.8 MB/s | 85 kB 00:00 2023-06-23T01:08:03.862 INFO:teuthology.orchestra.run.smithi134.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 15 MB/s | 70 kB 00:00 2023-06-23T01:08:03.878 INFO:teuthology.orchestra.run.smithi134.stdout:(70/120): python3-zc-lockfile-2.0-2.el8.noarch. 692 kB/s | 23 kB 00:00 2023-06-23T01:08:03.894 INFO:teuthology.orchestra.run.smithi134.stdout:(71/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 588 kB/s | 19 kB 00:00 2023-06-23T01:08:03.909 INFO:teuthology.orchestra.run.smithi134.stdout:(72/120): python3-jwt-1.6.1-2.el8.noarch.rpm 1.3 MB/s | 43 kB 00:00 2023-06-23T01:08:03.911 INFO:teuthology.orchestra.run.smithi134.stdout:(73/120): python3-oauthlib-2.1.0-1.el8.noarch.r 9.4 MB/s | 155 kB 00:00 2023-06-23T01:08:03.926 INFO:teuthology.orchestra.run.smithi134.stdout:(74/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 13 MB/s | 220 kB 00:00 2023-06-23T01:08:03.927 INFO:teuthology.orchestra.run.smithi134.stdout:(75/120): keyutils-libs-devel-1.5.10-6.el8.x86_ 2.8 MB/s | 48 kB 00:00 2023-06-23T01:08:03.942 INFO:teuthology.orchestra.run.smithi134.stdout:(76/120): python3-requests-oauthlib-1.0.0-1.el8 3.0 MB/s | 43 kB 00:00 2023-06-23T01:08:03.943 INFO:teuthology.orchestra.run.smithi134.stdout:(77/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 13 MB/s | 229 kB 00:00 2023-06-23T01:08:03.957 INFO:teuthology.orchestra.run.smithi134.stdout:(78/120): libverto-devel-0.3.0-5.el8.x86_64.rpm 1.3 MB/s | 18 kB 00:00 2023-06-23T01:08:03.971 INFO:teuthology.orchestra.run.smithi134.stdout:(79/120): thrift-0.13.0-2.el8.x86_64.rpm 15 MB/s | 1.7 MB 00:00 2023-06-23T01:08:03.985 INFO:teuthology.orchestra.run.smithi134.stdout:(80/120): pcre2-devel-10.32-2.el8.x86_64.rpm 21 MB/s | 605 kB 00:00 2023-06-23T01:08:03.999 INFO:teuthology.orchestra.run.smithi134.stdout:(81/120): openssl-devel-1.1.1g-15.el8_3.x86_64. 41 MB/s | 2.3 MB 00:00 2023-06-23T01:08:04.000 INFO:teuthology.orchestra.run.smithi134.stdout:(82/120): elfutils-libelf-devel-0.182-3.el8.x86 3.9 MB/s | 59 kB 00:00 2023-06-23T01:08:04.001 INFO:teuthology.orchestra.run.smithi134.stdout:(83/120): fuse-2.9.7-12.el8.x86_64.rpm 2.6 MB/s | 83 kB 00:00 2023-06-23T01:08:04.014 INFO:teuthology.orchestra.run.smithi134.stdout:(84/120): libcom_err-devel-1.45.6-1.el8.x86_64. 2.7 MB/s | 38 kB 00:00 2023-06-23T01:08:04.015 INFO:teuthology.orchestra.run.smithi134.stdout:(85/120): libsepol-devel-2.9-2.el8.x86_64.rpm 6.1 MB/s | 86 kB 00:00 2023-06-23T01:08:04.016 INFO:teuthology.orchestra.run.smithi134.stdout:(86/120): libquadmath-8.4.1-1.el8.x86_64.rpm 9.6 MB/s | 169 kB 00:00 2023-06-23T01:08:04.028 INFO:teuthology.orchestra.run.smithi134.stdout:(87/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 4.1 MB/s | 58 kB 00:00 2023-06-23T01:08:04.030 INFO:teuthology.orchestra.run.smithi134.stdout:(88/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 3.2 MB/s | 47 kB 00:00 2023-06-23T01:08:04.035 INFO:teuthology.orchestra.run.smithi134.stdout:(89/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.rp 29 MB/s | 559 kB 00:00 2023-06-23T01:08:04.045 INFO:teuthology.orchestra.run.smithi134.stdout:(90/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rpm 4.4 MB/s | 70 kB 00:00 2023-06-23T01:08:04.050 INFO:teuthology.orchestra.run.smithi134.stdout:(91/120): libgfortran-8.4.1-1.el8.x86_64.rpm 30 MB/s | 639 kB 00:00 2023-06-23T01:08:04.056 INFO:teuthology.orchestra.run.smithi134.stdout:(92/120): libselinux-devel-2.9-5.el8.x86_64.rpm 9.7 MB/s | 200 kB 00:00 2023-06-23T01:08:04.060 INFO:teuthology.orchestra.run.smithi134.stdout:(93/120): libuuid-devel-2.32.1-27.el8.x86_64.rp 6.4 MB/s | 97 kB 00:00 2023-06-23T01:08:04.071 INFO:teuthology.orchestra.run.smithi134.stdout:(94/120): python3-prettytable-0.7.2-14.el8.noar 3.0 MB/s | 44 kB 00:00 2023-06-23T01:08:04.074 INFO:teuthology.orchestra.run.smithi134.stdout:(95/120): python3-jsonpatch-1.21-2.el8.noarch.r 1.9 MB/s | 27 kB 00:00 2023-06-23T01:08:04.087 INFO:teuthology.orchestra.run.smithi134.stdout:(96/120): jq-1.5-12.el8.x86_64.rpm 9.7 MB/s | 161 kB 00:00 2023-06-23T01:08:04.093 INFO:teuthology.orchestra.run.smithi134.stdout:(97/120): flex-2.6.1-9.el8.x86_64.rpm 17 MB/s | 320 kB 00:00 2023-06-23T01:08:04.106 INFO:teuthology.orchestra.run.smithi134.stdout:(98/120): python3-werkzeug-0.12.2-4.el8.noarch. 24 MB/s | 457 kB 00:00 2023-06-23T01:08:04.107 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction test succeeded. 2023-06-23T01:08:04.111 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction 2023-06-23T01:08:04.119 INFO:teuthology.orchestra.run.smithi134.stdout:(99/120): python3-markupsafe-0.23-19.el8.x86_64 2.9 MB/s | 39 kB 00:00 2023-06-23T01:08:04.142 INFO:teuthology.orchestra.run.smithi134.stdout:(100/120): bison-3.0.4-10.el8.x86_64.rpm 30 MB/s | 688 kB 00:00 2023-06-23T01:08:04.158 INFO:teuthology.orchestra.run.smithi134.stdout:(101/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 16 MB/s | 259 kB 00:00 2023-06-23T01:08:04.175 INFO:teuthology.orchestra.run.smithi134.stdout:(102/120): python3-babel-2.5.1-5.el8.noarch.rpm 58 MB/s | 4.8 MB 00:00 2023-06-23T01:08:04.178 INFO:teuthology.orchestra.run.smithi134.stdout:(103/120): python3-numpy-f2py-1.14.3-9.el8.x86_ 11 MB/s | 225 kB 00:00 2023-06-23T01:08:04.190 INFO:teuthology.orchestra.run.smithi134.stdout:(104/120): python3-pyasn1-0.3.7-6.el8.noarch.rp 8.6 MB/s | 126 kB 00:00 2023-06-23T01:08:04.205 INFO:teuthology.orchestra.run.smithi134.stdout:(105/120): oniguruma-6.8.2-2.el8.x86_64.rpm 12 MB/s | 187 kB 00:00 2023-06-23T01:08:04.226 INFO:teuthology.orchestra.run.smithi134.stdout:(106/120): python3-jinja2-2.10.1-2.el8_0.noarch 25 MB/s | 538 kB 00:00 2023-06-23T01:08:04.338 INFO:teuthology.orchestra.run.smithi134.stdout:(107/120): python3-scipy-1.0.0-20.module+el8.1. 50 MB/s | 15 MB 00:00 2023-06-23T01:08:04.349 INFO:teuthology.orchestra.run.smithi134.stdout:(108/120): python3-numpy-1.14.3-9.el8.x86_64.rp 22 MB/s | 3.7 MB 00:00 2023-06-23T01:08:04.352 INFO:teuthology.orchestra.run.smithi134.stdout:(109/120): socat-1.7.3.3-2.el8.x86_64.rpm 2.3 MB/s | 302 kB 00:00 2023-06-23T01:08:04.353 INFO:teuthology.orchestra.run.smithi134.stdout:(110/120): python3-mako-1.0.6-13.el8.noarch.rpm 11 MB/s | 157 kB 00:00 2023-06-23T01:08:04.362 INFO:teuthology.orchestra.run.smithi134.stdout:(111/120): python3-pyasn1-modules-0.3.7-6.el8.n 7.8 MB/s | 110 kB 00:00 2023-06-23T01:08:04.368 INFO:teuthology.orchestra.run.smithi134.stdout:(112/120): python3-jsonpointer-1.10-11.el8.noar 1.3 MB/s | 20 kB 00:00 2023-06-23T01:08:04.371 INFO:teuthology.orchestra.run.smithi134.stdout:(113/120): librdkafka-0.11.4-1.el8.x86_64.rpm 19 MB/s | 353 kB 00:00 2023-06-23T01:08:04.383 INFO:teuthology.orchestra.run.smithi134.stdout:(114/120): libtool-2.4.6-25.el8.x86_64.rpm 35 MB/s | 709 kB 00:00 2023-06-23T01:08:04.384 INFO:teuthology.orchestra.run.smithi134.stdout:(115/120): python3-pytz-2017.2-9.el8.noarch.rpm 3.3 MB/s | 54 kB 00:00 2023-06-23T01:08:04.413 INFO:teuthology.orchestra.run.smithi134.stdout:(116/120): automake-1.16.1-7.el8.noarch.rpm 24 MB/s | 713 kB 00:00 2023-06-23T01:08:04.457 INFO:teuthology.orchestra.run.smithi134.stdout:(117/120): openblas-threads-0.3.12-1.el8.x86_64 55 MB/s | 4.7 MB 00:00 2023-06-23T01:08:04.470 INFO:teuthology.orchestra.run.smithi134.stdout:(118/120): openblas-0.3.12-1.el8.x86_64.rpm 53 MB/s | 4.6 MB 00:00 2023-06-23T01:08:04.888 INFO:teuthology.orchestra.run.smithi134.stdout:(119/120): librados2-17.2.6-719.g42ddc46e.el8.x 7.3 MB/s | 3.5 MB 00:00 2023-06-23T01:08:05.007 INFO:teuthology.orchestra.run.smithi134.stdout:(120/120): librbd1-17.2.6-719.g42ddc46e.el8.x86 6.5 MB/s | 3.6 MB 00:00 2023-06-23T01:08:05.010 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2023-06-23T01:08:05.010 INFO:teuthology.orchestra.run.smithi134.stdout:Total 11 MB/s | 201 MB 00:18 2023-06-23T01:08:05.057 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2023-06-23T01:08:05.168 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2023-06-23T01:08:05.168 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2023-06-23T01:08:06.584 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2023-06-23T01:08:06.599 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2023-06-23T01:08:06.629 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2023-06-23T01:08:06.845 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-06-23T01:08:07.554 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 2/122 2023-06-23T01:08:07.670 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 2/122 2023-06-23T01:08:07.862 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-06-23T01:08:08.275 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-06-23T01:08:08.503 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 4/122 2023-06-23T01:08:08.892 INFO:teuthology.orchestra.run.smithi120.stdout: Preparing : 1/1 2023-06-23T01:08:08.914 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 4/122 2023-06-23T01:08:09.091 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-06-23T01:08:09.128 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 5/122 2023-06-23T01:08:09.281 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 5/122 2023-06-23T01:08:09.303 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 2/122 2023-06-23T01:08:09.486 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 6/122 2023-06-23T01:08:09.490 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 3/122 2023-06-23T01:08:09.586 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 6/122 2023-06-23T01:08:09.621 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 3/122 2023-06-23T01:08:09.775 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rados-2:17.2.6-719.g42ddc46e.el8.x86_64 7/122 2023-06-23T01:08:09.832 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-06-23T01:08:10.100 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/122 2023-06-23T01:08:10.274 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-06-23T01:08:10.765 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 5/122 2023-06-23T01:08:10.777 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-06-23T01:08:10.961 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 10/122 2023-06-23T01:08:11.047 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2023-06-23T01:08:11.077 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 10/122 2023-06-23T01:08:11.222 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-06-23T01:08:11.236 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 5/122 2023-06-23T01:08:11.307 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-06-23T01:08:11.465 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 6/122 2023-06-23T01:08:11.484 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-06-23T01:08:11.623 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 6/122 2023-06-23T01:08:11.711 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 13/122 2023-06-23T01:08:11.799 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 13/122 2023-06-23T01:08:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 7/122 2023-06-23T01:08:11.839 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 2/122 2023-06-23T01:08:11.933 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 7/122 2023-06-23T01:08:11.952 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 2/122 2023-06-23T01:08:11.969 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_ 14/122 2023-06-23T01:08:12.028 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_ 14/122 2023-06-23T01:08:12.197 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-06-23T01:08:12.224 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 15/122 2023-06-23T01:08:12.283 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rados-2:17.2.6-719.g42ddc46e.el8.x86_64 8/122 2023-06-23T01:08:12.483 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 16/122 2023-06-23T01:08:12.567 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-06-23T01:08:12.755 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 4/122 2023-06-23T01:08:13.005 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-06-23T01:08:13.156 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 4/122 2023-06-23T01:08:13.169 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 17/122 2023-06-23T01:08:13.223 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 10/122 2023-06-23T01:08:13.303 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 18/122 2023-06-23T01:08:13.368 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 10/122 2023-06-23T01:08:13.368 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 5/122 2023-06-23T01:08:13.474 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 5/122 2023-06-23T01:08:13.486 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 19/122 2023-06-23T01:08:13.588 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-06-23T01:08:13.627 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 19/122 2023-06-23T01:08:13.647 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 6/122 2023-06-23T01:08:13.743 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-06-23T01:08:13.768 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 6/122 2023-06-23T01:08:13.786 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 20/122 2023-06-23T01:08:13.956 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 13/122 2023-06-23T01:08:13.969 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rados-2:17.2.6-719.g42ddc46e.el8.x86_64 7/122 2023-06-23T01:08:14.075 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 13/122 2023-06-23T01:08:14.240 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_ 14/122 2023-06-23T01:08:14.305 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/122 2023-06-23T01:08:14.325 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_ 14/122 2023-06-23T01:08:14.367 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 20/122 2023-06-23T01:08:14.475 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 15/122 2023-06-23T01:08:14.527 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-06-23T01:08:14.606 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 16/122 2023-06-23T01:08:14.783 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 16/122 2023-06-23T01:08:14.956 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 17/122 2023-06-23T01:08:14.962 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-06-23T01:08:15.061 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-06-23T01:08:15.130 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 10/122 2023-06-23T01:08:15.241 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 10/122 2023-06-23T01:08:15.302 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-0.3.12-1.el8.x86_64 22/122 2023-06-23T01:08:15.476 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-06-23T01:08:15.604 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 17/122 2023-06-23T01:08:15.667 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-06-23T01:08:15.781 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-06-23T01:08:15.855 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 13/122 2023-06-23T01:08:15.951 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 13/122 2023-06-23T01:08:16.016 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 22/122 2023-06-23T01:08:16.105 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_ 14/122 2023-06-23T01:08:16.163 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_ 14/122 2023-06-23T01:08:16.264 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 23/122 2023-06-23T01:08:16.343 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 15/122 2023-06-23T01:08:16.379 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-06-23T01:08:16.521 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 24/122 2023-06-23T01:08:16.547 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : openblas-0.3.12-1.el8.x86_64 19/122 2023-06-23T01:08:16.588 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 16/122 2023-06-23T01:08:16.738 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-06-23T01:08:16.803 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 17/122 2023-06-23T01:08:16.925 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-common-2:17.2.6-719.g42ddc46e.el8.x 26/122 2023-06-23T01:08:17.046 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-argparse-2:17.2.6-719.g42ddc46e.el8 27/122 2023-06-23T01:08:17.176 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 18/122 2023-06-23T01:08:17.305 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 19/122 2023-06-23T01:08:17.345 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cephfs-2:17.2.6-719.g42ddc46e.el8.x86_64 28/122 2023-06-23T01:08:17.424 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 19/122 2023-06-23T01:08:17.442 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 19/122 2023-06-23T01:08:17.566 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 29/122 2023-06-23T01:08:17.613 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 20/122 2023-06-23T01:08:17.698 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 20/122 2023-06-23T01:08:17.885 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 30/122 2023-06-23T01:08:17.891 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 21/122 2023-06-23T01:08:18.062 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 22/122 2023-06-23T01:08:18.164 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 20/122 2023-06-23T01:08:18.168 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-06-23T01:08:18.330 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-06-23T01:08:18.336 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/122 2023-06-23T01:08:18.653 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 24/122 2023-06-23T01:08:18.852 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-06-23T01:08:18.904 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-06-23T01:08:19.038 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-0.3.12-1.el8.x86_64 22/122 2023-06-23T01:08:19.072 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-ceph-common-2:17.2.6-719.g42ddc46e.el8.x 26/122 2023-06-23T01:08:19.202 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-ceph-argparse-2:17.2.6-719.g42ddc46e.el8 27/122 2023-06-23T01:08:19.443 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cephfs-2:17.2.6-719.g42ddc46e.el8.x86_64 28/122 2023-06-23T01:08:19.743 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 22/122 2023-06-23T01:08:20.007 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 23/122 2023-06-23T01:08:20.224 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-06-23T01:08:20.254 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 24/122 2023-06-23T01:08:20.515 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-06-23T01:08:20.568 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 29/122 2023-06-23T01:08:20.583 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 32/122 2023-06-23T01:08:20.688 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-common-2:17.2.6-719.g42ddc46e.el8.x 26/122 2023-06-23T01:08:20.798 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-argparse-2:17.2.6-719.g42ddc46e.el8 27/122 2023-06-23T01:08:20.919 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 30/122 2023-06-23T01:08:21.052 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cephfs-2:17.2.6-719.g42ddc46e.el8.x86_64 28/122 2023-06-23T01:08:21.197 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 31/122 2023-06-23T01:08:21.311 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 29/122 2023-06-23T01:08:21.442 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 32/122 2023-06-23T01:08:21.586 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 30/122 2023-06-23T01:08:21.601 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/122 2023-06-23T01:08:21.720 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 33/122 2023-06-23T01:08:21.787 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-06-23T01:08:21.930 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-06-23T01:08:21.939 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 34/122 2023-06-23T01:08:22.247 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 35/122 2023-06-23T01:08:22.448 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rsa-4.8-1.el8.noarch 36/122 2023-06-23T01:08:22.646 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-06-23T01:08:22.868 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rgw-2:17.2.6-719.g42ddc46e.el8.x86_64 38/122 2023-06-23T01:08:23.081 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rbd-2:17.2.6-719.g42ddc46e.el8.x86_64 39/122 2023-06-23T01:08:23.376 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librados-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 40/122 2023-06-23T01:08:23.608 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : automake-1.16.1-7.el8.noarch 41/122 2023-06-23T01:08:23.833 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 42/122 2023-06-23T01:08:23.849 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-06-23T01:08:24.031 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 43/122 2023-06-23T01:08:24.201 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 32/122 2023-06-23T01:08:24.246 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 44/122 2023-06-23T01:08:24.321 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-06-23T01:08:24.417 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 45/122 2023-06-23T01:08:24.484 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 45/122 2023-06-23T01:08:24.686 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : jq-1.5-12.el8.x86_64 46/122 2023-06-23T01:08:24.751 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 35/122 2023-06-23T01:08:24.860 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 47/122 2023-06-23T01:08:24.995 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rsa-4.8-1.el8.noarch 36/122 2023-06-23T01:08:25.022 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 48/122 2023-06-23T01:08:25.181 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 49/122 2023-06-23T01:08:25.184 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-06-23T01:08:25.185 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/122 2023-06-23T01:08:25.353 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 50/122 2023-06-23T01:08:25.384 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rgw-2:17.2.6-719.g42ddc46e.el8.x86_64 38/122 2023-06-23T01:08:25.530 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 34/122 2023-06-23T01:08:25.561 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rbd-2:17.2.6-719.g42ddc46e.el8.x86_64 39/122 2023-06-23T01:08:25.562 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : fuse-2.9.7-12.el8.x86_64 51/122 2023-06-23T01:08:25.689 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 52/122 2023-06-23T01:08:25.722 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librados-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 40/122 2023-06-23T01:08:25.756 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 35/122 2023-06-23T01:08:25.901 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 41/122 2023-06-23T01:08:25.950 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 53/122 2023-06-23T01:08:25.967 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rsa-4.8-1.el8.noarch 36/122 2023-06-23T01:08:26.085 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 42/122 2023-06-23T01:08:26.104 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 54/122 2023-06-23T01:08:26.148 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-06-23T01:08:26.249 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 43/122 2023-06-23T01:08:26.380 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rgw-2:17.2.6-719.g42ddc46e.el8.x86_64 38/122 2023-06-23T01:08:26.432 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : fuse-2.9.7-12.el8.x86_64 44/122 2023-06-23T01:08:26.470 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/122 2023-06-23T01:08:26.558 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rbd-2:17.2.6-719.g42ddc46e.el8.x86_64 39/122 2023-06-23T01:08:26.594 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 45/122 2023-06-23T01:08:26.730 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/122 2023-06-23T01:08:26.783 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 46/122 2023-06-23T01:08:26.819 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librados-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 40/122 2023-06-23T01:08:26.952 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 47/122 2023-06-23T01:08:26.992 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/122 2023-06-23T01:08:27.060 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : automake-1.16.1-7.el8.noarch 41/122 2023-06-23T01:08:27.252 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 42/122 2023-06-23T01:08:27.294 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 48/122 2023-06-23T01:08:27.298 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 58/122 2023-06-23T01:08:27.465 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 43/122 2023-06-23T01:08:27.467 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-06-23T01:08:27.612 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 49/122 2023-06-23T01:08:27.669 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 44/122 2023-06-23T01:08:27.671 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-06-23T01:08:27.878 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 45/122 2023-06-23T01:08:27.898 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 50/122 2023-06-23T01:08:27.930 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-06-23T01:08:27.960 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 45/122 2023-06-23T01:08:28.147 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : jq-1.5-12.el8.x86_64 46/122 2023-06-23T01:08:28.192 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-06-23T01:08:28.256 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 51/122 2023-06-23T01:08:28.304 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 47/122 2023-06-23T01:08:28.367 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-06-23T01:08:28.479 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : automake-1.16.1-7.el8.noarch 52/122 2023-06-23T01:08:28.482 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 48/122 2023-06-23T01:08:28.572 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-06-23T01:08:28.647 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 53/122 2023-06-23T01:08:28.650 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 49/122 2023-06-23T01:08:28.721 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-06-23T01:08:28.822 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 50/122 2023-06-23T01:08:28.848 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 54/122 2023-06-23T01:08:28.965 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-06-23T01:08:28.997 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : fuse-2.9.7-12.el8.x86_64 51/122 2023-06-23T01:08:29.035 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 55/122 2023-06-23T01:08:29.158 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 52/122 2023-06-23T01:08:29.165 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-06-23T01:08:29.179 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 56/122 2023-06-23T01:08:29.267 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 56/122 2023-06-23T01:08:29.369 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 53/122 2023-06-23T01:08:29.415 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-06-23T01:08:29.472 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : jq-1.5-12.el8.x86_64 57/122 2023-06-23T01:08:29.574 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 54/122 2023-06-23T01:08:29.595 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-06-23T01:08:29.658 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 58/122 2023-06-23T01:08:29.799 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-06-23T01:08:29.823 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-06-23T01:08:29.898 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/122 2023-06-23T01:08:29.997 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-06-23T01:08:30.054 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-06-23T01:08:30.165 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/122 2023-06-23T01:08:30.259 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-06-23T01:08:30.297 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-06-23T01:08:30.455 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/122 2023-06-23T01:08:30.481 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-06-23T01:08:30.490 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-06-23T01:08:30.744 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 58/122 2023-06-23T01:08:30.806 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-06-23T01:08:30.920 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-06-23T01:08:31.013 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-06-23T01:08:31.092 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-06-23T01:08:31.100 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-06-23T01:08:31.168 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-06-23T01:08:31.311 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-06-23T01:08:31.343 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-06-23T01:08:31.397 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-06-23T01:08:31.537 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-06-23T01:08:31.596 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-06-23T01:08:31.640 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-06-23T01:08:31.721 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-06-23T01:08:31.755 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-06-23T01:08:31.910 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-06-23T01:08:31.916 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-06-23T01:08:32.090 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-06-23T01:08:32.090 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-06-23T01:08:32.244 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-06-23T01:08:32.283 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-06-23T01:08:32.468 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-06-23T01:08:32.478 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-06-23T01:08:32.667 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-06-23T01:08:32.727 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-06-23T01:08:32.805 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-06-23T01:08:32.927 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-06-23T01:08:33.080 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-06-23T01:08:33.196 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-06-23T01:08:33.248 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-06-23T01:08:33.351 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-06-23T01:08:33.385 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-06-23T01:08:33.513 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-06-23T01:08:33.631 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-06-23T01:08:33.707 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-06-23T01:08:33.857 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-06-23T01:08:33.964 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-06-23T01:08:34.050 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-06-23T01:08:34.114 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-06-23T01:08:34.328 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-06-23T01:08:34.393 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-06-23T01:08:34.593 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-06-23T01:08:34.760 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-06-23T01:08:34.767 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-06-23T01:08:34.986 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-06-23T01:08:35.059 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-06-23T01:08:35.258 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-06-23T01:08:35.352 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-06-23T01:08:35.516 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-06-23T01:08:35.643 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-06-23T01:08:35.944 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-06-23T01:08:36.120 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-06-23T01:08:36.261 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-06-23T01:08:36.417 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-06-23T01:08:36.525 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-06-23T01:08:36.628 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-06-23T01:08:36.872 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-06-23T01:08:36.934 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-06-23T01:08:37.115 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-06-23T01:08:37.204 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-06-23T01:08:37.308 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-06-23T01:08:37.506 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-06-23T01:08:37.520 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-06-23T01:08:37.774 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-06-23T01:08:37.977 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-06-23T01:08:38.177 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-06-23T01:08:40.425 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 87/122 2023-06-23T01:08:40.673 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 87/122 2023-06-23T01:08:41.270 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 87/122 2023-06-23T01:08:41.549 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 88/122 2023-06-23T01:08:41.640 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 88/122 2023-06-23T01:08:41.781 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 89/122 2023-06-23T01:08:43.051 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 87/122 2023-06-23T01:08:43.284 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 87/122 2023-06-23T01:08:44.024 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 87/122 2023-06-23T01:08:44.025 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 87/122 2023-06-23T01:08:44.263 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 88/122 2023-06-23T01:08:44.294 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 87/122 2023-06-23T01:08:44.367 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 88/122 2023-06-23T01:08:44.475 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 89/122 2023-06-23T01:08:44.998 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 87/122 2023-06-23T01:08:45.288 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 88/122 2023-06-23T01:08:45.507 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 88/122 2023-06-23T01:08:45.643 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 89/122 2023-06-23T01:08:59.624 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 89/122 2023-06-23T01:08:59.624 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /sys 2023-06-23T01:08:59.624 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /proc 2023-06-23T01:08:59.624 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /dev 2023-06-23T01:08:59.625 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /run 2023-06-23T01:08:59.625 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /mnt 2023-06-23T01:08:59.625 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /var/tmp 2023-06-23T01:08:59.625 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /home 2023-06-23T01:08:59.625 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /tmp 2023-06-23T01:08:59.625 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /dev 2023-06-23T01:08:59.625 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-06-23T01:09:00.004 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 90/122 2023-06-23T01:09:01.481 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 90/122 2023-06-23T01:09:01.670 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 91/122 2023-06-23T01:09:01.891 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 91/122 2023-06-23T01:09:02.036 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 92/122 2023-06-23T01:09:02.423 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 92/122 2023-06-23T01:09:02.448 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 89/122 2023-06-23T01:09:02.448 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /sys 2023-06-23T01:09:02.449 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /proc 2023-06-23T01:09:02.449 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /dev 2023-06-23T01:09:02.449 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /run 2023-06-23T01:09:02.449 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /mnt 2023-06-23T01:09:02.449 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /var/tmp 2023-06-23T01:09:02.449 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /home 2023-06-23T01:09:02.449 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /tmp 2023-06-23T01:09:02.450 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /dev 2023-06-23T01:09:02.450 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-06-23T01:09:02.611 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 93/122 2023-06-23T01:09:02.628 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 93/122 2023-06-23T01:09:02.806 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 90/122 2023-06-23T01:09:04.306 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 90/122 2023-06-23T01:09:04.476 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 91/122 2023-06-23T01:09:04.708 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 91/122 2023-06-23T01:09:04.848 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 92/122 2023-06-23T01:09:05.182 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 89/122 2023-06-23T01:09:05.183 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /sys 2023-06-23T01:09:05.183 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /proc 2023-06-23T01:09:05.183 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /dev 2023-06-23T01:09:05.183 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /run 2023-06-23T01:09:05.184 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /mnt 2023-06-23T01:09:05.184 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /var/tmp 2023-06-23T01:09:05.184 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /home 2023-06-23T01:09:05.184 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /tmp 2023-06-23T01:09:05.184 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /dev 2023-06-23T01:09:05.184 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:09:05.235 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 92/122 2023-06-23T01:09:05.356 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 94/122 2023-06-23T01:09:05.381 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 93/122 2023-06-23T01:09:05.397 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 93/122 2023-06-23T01:09:05.582 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 94/122 2023-06-23T01:09:05.590 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 90/122 2023-06-23T01:09:05.725 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-719.g42ddc46e.el 95/122 2023-06-23T01:09:05.957 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-719.g42ddc46e.e 96/122 2023-06-23T01:09:06.127 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noar 97/122 2023-06-23T01:09:06.341 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noar 97/122 2023-06-23T01:09:06.552 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.no 98/122 2023-06-23T01:09:07.218 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 90/122 2023-06-23T01:09:07.393 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 91/122 2023-06-23T01:09:07.521 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.no 98/122 2023-06-23T01:09:07.672 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 91/122 2023-06-23T01:09:07.718 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-719.g42dd 99/122 2023-06-23T01:09:07.816 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-719.g42dd 99/122 2023-06-23T01:09:07.857 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 92/122 2023-06-23T01:09:08.014 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 94/122 2023-06-23T01:09:08.224 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 94/122 2023-06-23T01:09:08.232 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-719.g42ddc46e.el8 100/122 2023-06-23T01:09:08.310 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 92/122 2023-06-23T01:09:08.345 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-719.g42ddc46e.el 95/122 2023-06-23T01:09:08.436 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 101/122 2023-06-23T01:09:08.473 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 101/122 2023-06-23T01:09:08.481 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 93/122 2023-06-23T01:09:08.520 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 93/122 2023-06-23T01:09:08.568 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-719.g42ddc46e.e 96/122 2023-06-23T01:09:08.651 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 102/122 2023-06-23T01:09:08.663 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 102/122 2023-06-23T01:09:08.738 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noar 97/122 2023-06-23T01:09:08.949 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noar 97/122 2023-06-23T01:09:09.150 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.no 98/122 2023-06-23T01:09:10.115 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.no 98/122 2023-06-23T01:09:10.272 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-719.g42dd 99/122 2023-06-23T01:09:10.367 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-719.g42dd 99/122 2023-06-23T01:09:10.486 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-2:17.2.6-719.g42ddc46e.el8.x86_64 103/122 2023-06-23T01:09:10.713 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-719.g42ddc46e.el8 100/122 2023-06-23T01:09:10.719 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 104/122 2023-06-23T01:09:10.810 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 104/122 2023-06-23T01:09:10.867 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 101/122 2023-06-23T01:09:10.902 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 101/122 2023-06-23T01:09:11.010 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-719.g42ddc4 105/122 2023-06-23T01:09:11.053 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-719.g42ddc4 105/122 2023-06-23T01:09:11.080 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 102/122 2023-06-23T01:09:11.091 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 102/122 2023-06-23T01:09:11.251 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 106/122 2023-06-23T01:09:11.598 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 106/122 2023-06-23T01:09:11.776 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 107/122 2023-06-23T01:09:12.669 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 94/122 2023-06-23T01:09:12.669 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-2:17.2.6-719.g42ddc46e.el8.x86_64 103/122 2023-06-23T01:09:12.854 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 94/122 2023-06-23T01:09:12.867 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 104/122 2023-06-23T01:09:12.960 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 104/122 2023-06-23T01:09:13.064 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-719.g42ddc46e.el 95/122 2023-06-23T01:09:13.113 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-719.g42ddc4 105/122 2023-06-23T01:09:13.155 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-719.g42ddc4 105/122 2023-06-23T01:09:13.242 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-719.g42ddc46e.e 96/122 2023-06-23T01:09:13.338 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 106/122 2023-06-23T01:09:13.413 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noar 97/122 2023-06-23T01:09:13.677 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noar 97/122 2023-06-23T01:09:13.683 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 106/122 2023-06-23T01:09:13.846 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 107/122 2023-06-23T01:09:13.849 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.no 98/122 2023-06-23T01:09:14.962 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.no 98/122 2023-06-23T01:09:15.138 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-719.g42dd 99/122 2023-06-23T01:09:15.277 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-719.g42dd 99/122 2023-06-23T01:09:15.643 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-719.g42ddc46e.el8 100/122 2023-06-23T01:09:15.841 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 101/122 2023-06-23T01:09:15.910 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 101/122 2023-06-23T01:09:16.046 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 102/122 2023-06-23T01:09:16.072 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 102/122 2023-06-23T01:09:16.684 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 107/122 2023-06-23T01:09:17.653 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-test-2:17.2.6-719.g42ddc46e.el8.x86_64 108/122 2023-06-23T01:09:17.809 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-2:17.2.6-719.g42ddc46e.el8.x86_64 103/122 2023-06-23T01:09:18.020 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 109/122 2023-06-23T01:09:18.041 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 104/122 2023-06-23T01:09:18.167 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 104/122 2023-06-23T01:09:18.228 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 110/122 2023-06-23T01:09:18.347 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-719.g42ddc4 105/122 2023-06-23T01:09:18.424 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-719.g42ddc4 105/122 2023-06-23T01:09:18.449 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 111/122 2023-06-23T01:09:18.621 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 106/122 2023-06-23T01:09:18.641 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libtool-2.4.6-25.el8.x86_64 112/122 2023-06-23T01:09:18.653 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 112/122 2023-06-23T01:09:18.747 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 107/122 2023-06-23T01:09:18.804 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs-devel-2:17.2.6-719.g42ddc46e.el8.x86_6 113/122 2023-06-23T01:09:18.963 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-06-23T01:09:19.033 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 106/122 2023-06-23T01:09:19.150 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-top-2:17.2.6-719.g42ddc46e.el8.noarch 115/122 2023-06-23T01:09:19.205 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 107/122 2023-06-23T01:09:19.342 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 116/122 2023-06-23T01:09:19.627 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-nbd-2:17.2.6-719.g42ddc46e.el8.x86_64 117/122 2023-06-23T01:09:19.663 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-test-2:17.2.6-719.g42ddc46e.el8.x86_64 108/122 2023-06-23T01:09:19.874 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : bison-3.0.4-10.el8.x86_64 118/122 2023-06-23T01:09:19.945 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 118/122 2023-06-23T01:09:20.032 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 109/122 2023-06-23T01:09:20.141 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flex-2.6.1-9.el8.x86_64 119/122 2023-06-23T01:09:20.152 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 119/122 2023-06-23T01:09:20.240 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 110/122 2023-06-23T01:09:20.310 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 120/122 2023-06-23T01:09:20.311 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-06-23T01:09:20.480 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 111/122 2023-06-23T01:09:20.511 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-06-23T01:09:20.511 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-06-23T01:09:20.636 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libtool-2.4.6-25.el8.x86_64 112/122 2023-06-23T01:09:20.647 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 112/122 2023-06-23T01:09:20.791 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs-devel-2:17.2.6-719.g42ddc46e.el8.x86_6 113/122 2023-06-23T01:09:20.958 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-06-23T01:09:21.073 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-06-23T01:09:21.073 INFO:teuthology.orchestra.run.smithi157.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-06-23T01:09:21.074 INFO:teuthology.orchestra.run.smithi157.stdout:All rules containing unresolvable specifiers will be skipped. 2023-06-23T01:09:21.074 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-06-23T01:09:21.095 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-top-2:17.2.6-719.g42ddc46e.el8.noarch 115/122 2023-06-23T01:09:21.264 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 116/122 2023-06-23T01:09:21.445 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-2:17.2.6-719.g42ddc46e.el8.x86_64 1/122 2023-06-23T01:09:21.445 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 2/122 2023-06-23T01:09:21.446 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 3/122 2023-06-23T01:09:21.446 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 4/122 2023-06-23T01:09:21.446 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-719.g42ddc4 5/122 2023-06-23T01:09:21.446 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 6/122 2023-06-23T01:09:21.446 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 7/122 2023-06-23T01:09:21.446 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 8/122 2023-06-23T01:09:21.446 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 9/122 2023-06-23T01:09:21.454 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 10/122 2023-06-23T01:09:21.454 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 11/122 2023-06-23T01:09:21.454 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-test-2:17.2.6-719.g42ddc46e.el8.x86_64 12/122 2023-06-23T01:09:21.455 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 13/122 2023-06-23T01:09:21.455 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs-devel-2:17.2.6-719.g42ddc46e.el8.x86_6 14/122 2023-06-23T01:09:21.455 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 15/122 2023-06-23T01:09:21.455 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 16/122 2023-06-23T01:09:21.455 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 17/122 2023-06-23T01:09:21.455 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_ 18/122 2023-06-23T01:09:21.456 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 19/122 2023-06-23T01:09:21.456 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-argparse-2:17.2.6-719.g42ddc46e.el8 20/122 2023-06-23T01:09:21.456 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-common-2:17.2.6-719.g42ddc46e.el8.x 21/122 2023-06-23T01:09:21.456 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cephfs-2:17.2.6-719.g42ddc46e.el8.x86_64 22/122 2023-06-23T01:09:21.456 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rados-2:17.2.6-719.g42ddc46e.el8.x86_64 23/122 2023-06-23T01:09:21.456 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rbd-2:17.2.6-719.g42ddc46e.el8.x86_64 24/122 2023-06-23T01:09:21.456 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rgw-2:17.2.6-719.g42ddc46e.el8.x86_64 25/122 2023-06-23T01:09:21.456 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 26/122 2023-06-23T01:09:21.457 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 27/122 2023-06-23T01:09:21.457 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-nbd-2:17.2.6-719.g42ddc46e.el8.x86_64 28/122 2023-06-23T01:09:21.457 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-719.g42ddc46e.e 29/122 2023-06-23T01:09:21.457 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noar 30/122 2023-06-23T01:09:21.457 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.no 31/122 2023-06-23T01:09:21.457 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-719.g42dd 32/122 2023-06-23T01:09:21.457 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-719.g42ddc46e.el8 33/122 2023-06-23T01:09:21.458 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 34/122 2023-06-23T01:09:21.458 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-719.g42ddc46e.el 35/122 2023-06-23T01:09:21.458 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 36/122 2023-06-23T01:09:21.458 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 37/122 2023-06-23T01:09:21.458 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-top-2:17.2.6-719.g42ddc46e.el8.noarch 38/122 2023-06-23T01:09:21.458 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-06-23T01:09:21.458 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-06-23T01:09:21.458 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-06-23T01:09:21.459 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-06-23T01:09:21.459 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-06-23T01:09:21.459 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-06-23T01:09:21.459 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-06-23T01:09:21.459 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-06-23T01:09:21.459 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-06-23T01:09:21.459 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-06-23T01:09:21.460 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-06-23T01:09:21.460 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-06-23T01:09:21.460 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-06-23T01:09:21.460 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-06-23T01:09:21.460 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-06-23T01:09:21.460 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-06-23T01:09:21.460 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-06-23T01:09:21.460 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-06-23T01:09:21.461 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-06-23T01:09:21.461 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-06-23T01:09:21.461 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-06-23T01:09:21.461 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/122 2023-06-23T01:09:21.461 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-06-23T01:09:21.461 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-06-23T01:09:21.461 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-06-23T01:09:21.461 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-06-23T01:09:21.462 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-06-23T01:09:21.462 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-06-23T01:09:21.462 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-06-23T01:09:21.462 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-06-23T01:09:21.462 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-06-23T01:09:21.462 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-06-23T01:09:21.462 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-06-23T01:09:21.463 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 72/122 2023-06-23T01:09:21.463 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/122 2023-06-23T01:09:21.463 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/122 2023-06-23T01:09:21.463 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/122 2023-06-23T01:09:21.463 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/122 2023-06-23T01:09:21.463 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 77/122 2023-06-23T01:09:21.463 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/122 2023-06-23T01:09:21.463 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 79/122 2023-06-23T01:09:21.464 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 80/122 2023-06-23T01:09:21.464 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 81/122 2023-06-23T01:09:21.464 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 82/122 2023-06-23T01:09:21.464 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 83/122 2023-06-23T01:09:21.464 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 84/122 2023-06-23T01:09:21.464 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 85/122 2023-06-23T01:09:21.464 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 86/122 2023-06-23T01:09:21.464 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 87/122 2023-06-23T01:09:21.465 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 88/122 2023-06-23T01:09:21.465 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 89/122 2023-06-23T01:09:21.465 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 90/122 2023-06-23T01:09:21.465 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 91/122 2023-06-23T01:09:21.466 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 92/122 2023-06-23T01:09:21.466 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 93/122 2023-06-23T01:09:21.466 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 94/122 2023-06-23T01:09:21.466 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 95/122 2023-06-23T01:09:21.466 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 96/122 2023-06-23T01:09:21.466 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : jq-1.5-12.el8.x86_64 97/122 2023-06-23T01:09:21.466 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flex-2.6.1-9.el8.x86_64 98/122 2023-06-23T01:09:21.467 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 99/122 2023-06-23T01:09:21.467 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 100/122 2023-06-23T01:09:21.467 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 101/122 2023-06-23T01:09:21.467 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : bison-3.0.4-10.el8.x86_64 102/122 2023-06-23T01:09:21.467 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 103/122 2023-06-23T01:09:21.467 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 104/122 2023-06-23T01:09:21.467 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 105/122 2023-06-23T01:09:21.467 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 106/122 2023-06-23T01:09:21.468 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/122 2023-06-23T01:09:21.468 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 108/122 2023-06-23T01:09:21.468 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 109/122 2023-06-23T01:09:21.468 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 110/122 2023-06-23T01:09:21.468 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 111/122 2023-06-23T01:09:21.468 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 112/122 2023-06-23T01:09:21.468 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 113/122 2023-06-23T01:09:21.469 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 114/122 2023-06-23T01:09:21.469 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 115/122 2023-06-23T01:09:21.469 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 116/122 2023-06-23T01:09:21.469 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 117/122 2023-06-23T01:09:21.469 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : automake-1.16.1-7.el8.noarch 118/122 2023-06-23T01:09:21.469 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 119/122 2023-06-23T01:09:21.469 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-06-23T01:09:21.469 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 121/122 2023-06-23T01:09:21.512 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-nbd-2:17.2.6-719.g42ddc46e.el8.x86_64 117/122 2023-06-23T01:09:21.687 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : bison-3.0.4-10.el8.x86_64 118/122 2023-06-23T01:09:21.745 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 118/122 2023-06-23T01:09:21.920 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flex-2.6.1-9.el8.x86_64 119/122 2023-06-23T01:09:21.928 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 119/122 2023-06-23T01:09:22.047 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 120/122 2023-06-23T01:09:22.047 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-06-23T01:09:22.239 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-06-23T01:09:22.239 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-06-23T01:09:22.736 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-06-23T01:09:22.736 INFO:teuthology.orchestra.run.smithi134.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-06-23T01:09:22.736 INFO:teuthology.orchestra.run.smithi134.stdout:All rules containing unresolvable specifiers will be skipped. 2023-06-23T01:09:22.737 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-06-23T01:09:23.082 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-2:17.2.6-719.g42ddc46e.el8.x86_64 1/122 2023-06-23T01:09:23.083 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 2/122 2023-06-23T01:09:23.083 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 3/122 2023-06-23T01:09:23.083 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 4/122 2023-06-23T01:09:23.083 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-719.g42ddc4 5/122 2023-06-23T01:09:23.083 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 6/122 2023-06-23T01:09:23.083 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 7/122 2023-06-23T01:09:23.084 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 8/122 2023-06-23T01:09:23.084 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 9/122 2023-06-23T01:09:23.084 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 10/122 2023-06-23T01:09:23.084 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 11/122 2023-06-23T01:09:23.084 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-test-2:17.2.6-719.g42ddc46e.el8.x86_64 12/122 2023-06-23T01:09:23.084 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 13/122 2023-06-23T01:09:23.084 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs-devel-2:17.2.6-719.g42ddc46e.el8.x86_6 14/122 2023-06-23T01:09:23.085 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 15/122 2023-06-23T01:09:23.085 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 16/122 2023-06-23T01:09:23.085 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 17/122 2023-06-23T01:09:23.085 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_ 18/122 2023-06-23T01:09:23.085 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 19/122 2023-06-23T01:09:23.085 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-argparse-2:17.2.6-719.g42ddc46e.el8 20/122 2023-06-23T01:09:23.085 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-common-2:17.2.6-719.g42ddc46e.el8.x 21/122 2023-06-23T01:09:23.085 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cephfs-2:17.2.6-719.g42ddc46e.el8.x86_64 22/122 2023-06-23T01:09:23.086 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rados-2:17.2.6-719.g42ddc46e.el8.x86_64 23/122 2023-06-23T01:09:23.086 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rbd-2:17.2.6-719.g42ddc46e.el8.x86_64 24/122 2023-06-23T01:09:23.086 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rgw-2:17.2.6-719.g42ddc46e.el8.x86_64 25/122 2023-06-23T01:09:23.092 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 26/122 2023-06-23T01:09:23.092 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 27/122 2023-06-23T01:09:23.092 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-nbd-2:17.2.6-719.g42ddc46e.el8.x86_64 28/122 2023-06-23T01:09:23.092 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-719.g42ddc46e.e 29/122 2023-06-23T01:09:23.093 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noar 30/122 2023-06-23T01:09:23.093 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.no 31/122 2023-06-23T01:09:23.093 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-719.g42dd 32/122 2023-06-23T01:09:23.093 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-719.g42ddc46e.el8 33/122 2023-06-23T01:09:23.093 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 34/122 2023-06-23T01:09:23.094 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-719.g42ddc46e.el 35/122 2023-06-23T01:09:23.094 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 36/122 2023-06-23T01:09:23.094 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 37/122 2023-06-23T01:09:23.094 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-top-2:17.2.6-719.g42ddc46e.el8.noarch 38/122 2023-06-23T01:09:23.094 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-06-23T01:09:23.094 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-06-23T01:09:23.094 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-06-23T01:09:23.095 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-06-23T01:09:23.095 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-06-23T01:09:23.095 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-06-23T01:09:23.095 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-06-23T01:09:23.095 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-06-23T01:09:23.095 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-06-23T01:09:23.095 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-06-23T01:09:23.095 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-06-23T01:09:23.096 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-06-23T01:09:23.096 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-06-23T01:09:23.096 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-06-23T01:09:23.096 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-06-23T01:09:23.096 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-06-23T01:09:23.096 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-06-23T01:09:23.096 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-06-23T01:09:23.096 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-06-23T01:09:23.097 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-06-23T01:09:23.097 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-06-23T01:09:23.097 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/122 2023-06-23T01:09:23.097 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-06-23T01:09:23.097 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-06-23T01:09:23.097 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-06-23T01:09:23.097 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-06-23T01:09:23.097 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-06-23T01:09:23.098 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-06-23T01:09:23.098 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-06-23T01:09:23.098 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-06-23T01:09:23.098 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-06-23T01:09:23.098 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-06-23T01:09:23.098 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-06-23T01:09:23.098 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 72/122 2023-06-23T01:09:23.099 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/122 2023-06-23T01:09:23.099 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/122 2023-06-23T01:09:23.099 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/122 2023-06-23T01:09:23.099 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/122 2023-06-23T01:09:23.099 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 77/122 2023-06-23T01:09:23.099 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/122 2023-06-23T01:09:23.099 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 79/122 2023-06-23T01:09:23.099 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 80/122 2023-06-23T01:09:23.100 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 81/122 2023-06-23T01:09:23.100 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 82/122 2023-06-23T01:09:23.100 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 83/122 2023-06-23T01:09:23.100 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 84/122 2023-06-23T01:09:23.100 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 85/122 2023-06-23T01:09:23.100 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 86/122 2023-06-23T01:09:23.100 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 87/122 2023-06-23T01:09:23.100 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 88/122 2023-06-23T01:09:23.101 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 89/122 2023-06-23T01:09:23.101 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 90/122 2023-06-23T01:09:23.101 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 91/122 2023-06-23T01:09:23.101 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 92/122 2023-06-23T01:09:23.101 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 93/122 2023-06-23T01:09:23.101 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 94/122 2023-06-23T01:09:23.101 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 95/122 2023-06-23T01:09:23.101 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 96/122 2023-06-23T01:09:23.102 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : jq-1.5-12.el8.x86_64 97/122 2023-06-23T01:09:23.102 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flex-2.6.1-9.el8.x86_64 98/122 2023-06-23T01:09:23.102 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 99/122 2023-06-23T01:09:23.102 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 100/122 2023-06-23T01:09:23.102 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 101/122 2023-06-23T01:09:23.102 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : bison-3.0.4-10.el8.x86_64 102/122 2023-06-23T01:09:23.102 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 103/122 2023-06-23T01:09:23.102 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 104/122 2023-06-23T01:09:23.103 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 105/122 2023-06-23T01:09:23.103 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 106/122 2023-06-23T01:09:23.103 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/122 2023-06-23T01:09:23.103 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 108/122 2023-06-23T01:09:23.103 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 109/122 2023-06-23T01:09:23.103 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 110/122 2023-06-23T01:09:23.103 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 111/122 2023-06-23T01:09:23.103 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 112/122 2023-06-23T01:09:23.104 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 113/122 2023-06-23T01:09:23.104 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 114/122 2023-06-23T01:09:23.104 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 115/122 2023-06-23T01:09:23.104 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 116/122 2023-06-23T01:09:23.104 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 117/122 2023-06-23T01:09:23.104 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : automake-1.16.1-7.el8.noarch 118/122 2023-06-23T01:09:23.104 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 119/122 2023-06-23T01:09:23.105 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-06-23T01:09:23.105 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 121/122 2023-06-23T01:09:23.650 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-06-23T01:09:23.650 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:46 ago on Fri 23 Jun 2023 01:07:37 AM UTC. 2023-06-23T01:09:24.612 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 107/122 2023-06-23T01:09:24.937 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-test-2:17.2.6-719.g42ddc46e.el8.x86_64 108/122 2023-06-23T01:09:25.111 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libtool-2.4.6-25.el8.x86_64 109/122 2023-06-23T01:09:25.228 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-06-23T01:09:25.228 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:01:47 ago on Fri 23 Jun 2023 01:07:38 AM UTC. 2023-06-23T01:09:25.908 INFO:teuthology.orchestra.run.smithi157.stdout:Installed products updated. 2023-06-23T01:09:26.030 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-06-23T01:09:26.030 INFO:teuthology.orchestra.run.smithi157.stdout:Upgraded: 2023-06-23T01:09:26.030 INFO:teuthology.orchestra.run.smithi157.stdout: librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:26.030 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:26.031 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2023-06-23T01:09:26.031 INFO:teuthology.orchestra.run.smithi157.stdout: automake-1.16.1-7.el8.noarch 2023-06-23T01:09:26.031 INFO:teuthology.orchestra.run.smithi157.stdout: bison-3.0.4-10.el8.x86_64 2023-06-23T01:09:26.031 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:26.031 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:26.031 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:26.031 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:26.032 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:09:26.032 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:26.032 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:26.032 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:26.032 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:09:26.032 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:09:26.032 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:09:26.033 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:09:26.033 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:09:26.033 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:26.033 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:26.033 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:09:26.033 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:26.033 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:26.033 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:26.034 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:09:26.034 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:09:26.034 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:26.034 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-top-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:09:26.034 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-06-23T01:09:26.034 INFO:teuthology.orchestra.run.smithi157.stdout: flex-2.6.1-9.el8.x86_64 2023-06-23T01:09:26.034 INFO:teuthology.orchestra.run.smithi157.stdout: fuse-2.9.7-12.el8.x86_64 2023-06-23T01:09:26.034 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-06-23T01:09:26.035 INFO:teuthology.orchestra.run.smithi157.stdout: jq-1.5-12.el8.x86_64 2023-06-23T01:09:26.035 INFO:teuthology.orchestra.run.smithi157.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-06-23T01:09:26.035 INFO:teuthology.orchestra.run.smithi157.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-06-23T01:09:26.035 INFO:teuthology.orchestra.run.smithi157.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-06-23T01:09:26.035 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:26.035 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:26.035 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:26.036 INFO:teuthology.orchestra.run.smithi157.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-06-23T01:09:26.036 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-06-23T01:09:26.036 INFO:teuthology.orchestra.run.smithi157.stdout: liboath-2.6.2-3.el8.x86_64 2023-06-23T01:09:26.036 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-06-23T01:09:26.036 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-06-23T01:09:26.036 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:26.036 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:26.037 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-06-23T01:09:26.037 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:26.037 INFO:teuthology.orchestra.run.smithi157.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-06-23T01:09:26.037 INFO:teuthology.orchestra.run.smithi157.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-06-23T01:09:26.037 INFO:teuthology.orchestra.run.smithi157.stdout: libtool-2.4.6-25.el8.x86_64 2023-06-23T01:09:26.038 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind-1.3.1-3.el8.x86_64 2023-06-23T01:09:26.038 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-06-23T01:09:26.038 INFO:teuthology.orchestra.run.smithi157.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-06-23T01:09:26.038 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-06-23T01:09:26.038 INFO:teuthology.orchestra.run.smithi157.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-06-23T01:09:26.038 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-0.3.12-1.el8.x86_64 2023-06-23T01:09:26.038 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-06-23T01:09:26.039 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-06-23T01:09:26.039 INFO:teuthology.orchestra.run.smithi157.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-06-23T01:09:26.039 INFO:teuthology.orchestra.run.smithi157.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-06-23T01:09:26.039 INFO:teuthology.orchestra.run.smithi157.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-06-23T01:09:26.039 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-06-23T01:09:26.039 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel-2.5.1-5.el8.noarch 2023-06-23T01:09:26.039 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-06-23T01:09:26.039 INFO:teuthology.orchestra.run.smithi157.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-06-23T01:09:26.040 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-06-23T01:09:26.040 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:26.040 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:26.040 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:26.040 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-06-23T01:09:26.040 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-06-23T01:09:26.040 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-06-23T01:09:26.041 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-06-23T01:09:26.041 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-6.2-6.el8.noarch 2023-06-23T01:09:26.041 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-06-23T01:09:26.041 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-06-23T01:09:26.041 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-06-23T01:09:26.041 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-06-23T01:09:26.041 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-06-23T01:09:26.041 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-06-23T01:09:26.042 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-06-23T01:09:26.042 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako-1.0.6-13.el8.noarch 2023-06-23T01:09:26.042 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-06-23T01:09:26.042 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-06-23T01:09:26.042 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-06-23T01:09:26.042 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-06-23T01:09:26.042 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-06-23T01:09:26.043 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-06-23T01:09:26.043 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-06-23T01:09:26.043 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend-2.6-1.el8.noarch 2023-06-23T01:09:26.043 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-06-23T01:09:26.043 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-06-23T01:09:26.043 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-06-23T01:09:26.043 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz-2017.2-9.el8.noarch 2023-06-23T01:09:26.043 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:26.044 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:26.044 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-06-23T01:09:26.044 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-06-23T01:09:26.044 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:26.044 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes-2.4.1-12.el8.noarch 2023-06-23T01:09:26.044 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa-4.8-1.el8.noarch 2023-06-23T01:09:26.044 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-06-23T01:09:26.045 INFO:teuthology.orchestra.run.smithi157.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-06-23T01:09:26.045 INFO:teuthology.orchestra.run.smithi157.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-06-23T01:09:26.045 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-06-23T01:09:26.045 INFO:teuthology.orchestra.run.smithi157.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-06-23T01:09:26.045 INFO:teuthology.orchestra.run.smithi157.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-06-23T01:09:26.045 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-06-23T01:09:26.045 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-06-23T01:09:26.046 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-06-23T01:09:26.046 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-06-23T01:09:26.046 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-06-23T01:09:26.046 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:26.046 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:26.046 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:26.046 INFO:teuthology.orchestra.run.smithi157.stdout: socat-1.7.3.3-2.el8.x86_64 2023-06-23T01:09:26.047 INFO:teuthology.orchestra.run.smithi157.stdout: thrift-0.13.0-2.el8.x86_64 2023-06-23T01:09:26.047 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-06-23T01:09:26.047 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-06-23T01:09:26.047 INFO:teuthology.orchestra.run.smithi157.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-06-23T01:09:26.048 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-06-23T01:09:26.048 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-06-23T01:09:26.182 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 109/122 2023-06-23T01:09:26.257 DEBUG:teuthology.parallel:result is None 2023-06-23T01:09:26.560 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 110/122 2023-06-23T01:09:26.752 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 111/122 2023-06-23T01:09:26.932 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 112/122 2023-06-23T01:09:27.076 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libcephfs-devel-2:17.2.6-719.g42ddc46e.el8.x86_6 113/122 2023-06-23T01:09:27.226 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-06-23T01:09:27.328 INFO:teuthology.orchestra.run.smithi134.stdout:Installed products updated. 2023-06-23T01:09:27.379 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephfs-top-2:17.2.6-719.g42ddc46e.el8.noarch 115/122 2023-06-23T01:09:27.447 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-06-23T01:09:27.448 INFO:teuthology.orchestra.run.smithi134.stdout:Upgraded: 2023-06-23T01:09:27.448 INFO:teuthology.orchestra.run.smithi134.stdout: librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:27.449 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:27.449 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2023-06-23T01:09:27.449 INFO:teuthology.orchestra.run.smithi134.stdout: automake-1.16.1-7.el8.noarch 2023-06-23T01:09:27.450 INFO:teuthology.orchestra.run.smithi134.stdout: bison-3.0.4-10.el8.x86_64 2023-06-23T01:09:27.450 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:27.450 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:27.450 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:27.451 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:27.451 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:09:27.451 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:27.451 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:27.452 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:27.452 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:09:27.452 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:09:27.452 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:09:27.452 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:09:27.453 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:09:27.453 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:27.454 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:27.454 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:09:27.454 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:27.454 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:27.455 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:27.455 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:09:27.455 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:09:27.455 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:27.455 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:09:27.455 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-06-23T01:09:27.456 INFO:teuthology.orchestra.run.smithi134.stdout: flex-2.6.1-9.el8.x86_64 2023-06-23T01:09:27.456 INFO:teuthology.orchestra.run.smithi134.stdout: fuse-2.9.7-12.el8.x86_64 2023-06-23T01:09:27.456 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-06-23T01:09:27.456 INFO:teuthology.orchestra.run.smithi134.stdout: jq-1.5-12.el8.x86_64 2023-06-23T01:09:27.456 INFO:teuthology.orchestra.run.smithi134.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-06-23T01:09:27.456 INFO:teuthology.orchestra.run.smithi134.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-06-23T01:09:27.456 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-06-23T01:09:27.457 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:27.457 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:27.457 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:27.457 INFO:teuthology.orchestra.run.smithi134.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-06-23T01:09:27.457 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-06-23T01:09:27.457 INFO:teuthology.orchestra.run.smithi134.stdout: liboath-2.6.2-3.el8.x86_64 2023-06-23T01:09:27.457 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-06-23T01:09:27.457 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-06-23T01:09:27.458 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:27.458 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:27.458 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-06-23T01:09:27.458 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:27.458 INFO:teuthology.orchestra.run.smithi134.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-06-23T01:09:27.458 INFO:teuthology.orchestra.run.smithi134.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-06-23T01:09:27.458 INFO:teuthology.orchestra.run.smithi134.stdout: libtool-2.4.6-25.el8.x86_64 2023-06-23T01:09:27.458 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind-1.3.1-3.el8.x86_64 2023-06-23T01:09:27.459 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-06-23T01:09:27.459 INFO:teuthology.orchestra.run.smithi134.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-06-23T01:09:27.459 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-06-23T01:09:27.459 INFO:teuthology.orchestra.run.smithi134.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-06-23T01:09:27.459 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-0.3.12-1.el8.x86_64 2023-06-23T01:09:27.459 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-06-23T01:09:27.459 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-06-23T01:09:27.460 INFO:teuthology.orchestra.run.smithi134.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-06-23T01:09:27.460 INFO:teuthology.orchestra.run.smithi134.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-06-23T01:09:27.460 INFO:teuthology.orchestra.run.smithi134.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-06-23T01:09:27.460 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-06-23T01:09:27.460 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel-2.5.1-5.el8.noarch 2023-06-23T01:09:27.460 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-06-23T01:09:27.461 INFO:teuthology.orchestra.run.smithi134.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-06-23T01:09:27.461 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-06-23T01:09:27.461 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:27.461 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:27.461 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:27.461 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-06-23T01:09:27.461 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-06-23T01:09:27.461 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-06-23T01:09:27.462 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-06-23T01:09:27.462 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-6.2-6.el8.noarch 2023-06-23T01:09:27.462 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-06-23T01:09:27.462 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-06-23T01:09:27.462 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-06-23T01:09:27.462 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-06-23T01:09:27.462 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-06-23T01:09:27.462 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-06-23T01:09:27.463 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-06-23T01:09:27.463 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako-1.0.6-13.el8.noarch 2023-06-23T01:09:27.463 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-06-23T01:09:27.463 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-06-23T01:09:27.463 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-06-23T01:09:27.463 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-06-23T01:09:27.463 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-06-23T01:09:27.463 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-06-23T01:09:27.464 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-06-23T01:09:27.464 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend-2.6-1.el8.noarch 2023-06-23T01:09:27.464 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-06-23T01:09:27.464 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-06-23T01:09:27.464 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-06-23T01:09:27.464 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz-2017.2-9.el8.noarch 2023-06-23T01:09:27.464 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:27.465 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:27.465 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-06-23T01:09:27.465 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-06-23T01:09:27.465 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:27.465 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes-2.4.1-12.el8.noarch 2023-06-23T01:09:27.465 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa-4.8-1.el8.noarch 2023-06-23T01:09:27.466 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-06-23T01:09:27.466 INFO:teuthology.orchestra.run.smithi134.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-06-23T01:09:27.466 INFO:teuthology.orchestra.run.smithi134.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-06-23T01:09:27.466 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-06-23T01:09:27.466 INFO:teuthology.orchestra.run.smithi134.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-06-23T01:09:27.467 INFO:teuthology.orchestra.run.smithi134.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-06-23T01:09:27.468 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-06-23T01:09:27.468 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-06-23T01:09:27.468 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-06-23T01:09:27.468 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-06-23T01:09:27.469 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-06-23T01:09:27.469 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:27.469 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:27.469 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:27.470 INFO:teuthology.orchestra.run.smithi134.stdout: socat-1.7.3.3-2.el8.x86_64 2023-06-23T01:09:27.470 INFO:teuthology.orchestra.run.smithi134.stdout: thrift-0.13.0-2.el8.x86_64 2023-06-23T01:09:27.470 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-06-23T01:09:27.471 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-06-23T01:09:27.472 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-06-23T01:09:27.472 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-06-23T01:09:27.472 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2023-06-23T01:09:27.583 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : rbd-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 116/122 2023-06-23T01:09:27.669 DEBUG:teuthology.parallel:result is None 2023-06-23T01:09:27.746 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : rbd-nbd-2:17.2.6-719.g42ddc46e.el8.x86_64 117/122 2023-06-23T01:09:27.972 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 118/122 2023-06-23T01:09:28.161 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : bison-3.0.4-10.el8.x86_64 119/122 2023-06-23T01:09:28.255 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 119/122 2023-06-23T01:09:28.428 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flex-2.6.1-9.el8.x86_64 120/122 2023-06-23T01:09:28.451 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 120/122 2023-06-23T01:09:28.451 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-06-23T01:09:28.667 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-06-23T01:09:28.668 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-06-23T01:09:29.044 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-06-23T01:09:29.045 INFO:teuthology.orchestra.run.smithi120.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-06-23T01:09:29.045 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:09:29.487 INFO:teuthology.orchestra.run.smithi120.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-06-23T01:09:29.487 INFO:teuthology.orchestra.run.smithi120.stdout:All rules containing unresolvable specifiers will be skipped. 2023-06-23T01:09:29.487 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:09:29.885 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-2:17.2.6-719.g42ddc46e.el8.x86_64 1/122 2023-06-23T01:09:29.886 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 2/122 2023-06-23T01:09:29.886 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 3/122 2023-06-23T01:09:29.886 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 4/122 2023-06-23T01:09:29.886 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-719.g42ddc4 5/122 2023-06-23T01:09:29.886 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 6/122 2023-06-23T01:09:29.886 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 7/122 2023-06-23T01:09:29.886 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 8/122 2023-06-23T01:09:29.887 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 9/122 2023-06-23T01:09:29.887 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 10/122 2023-06-23T01:09:29.887 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 11/122 2023-06-23T01:09:29.887 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-test-2:17.2.6-719.g42ddc46e.el8.x86_64 12/122 2023-06-23T01:09:29.887 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 13/122 2023-06-23T01:09:29.887 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libcephfs-devel-2:17.2.6-719.g42ddc46e.el8.x86_6 14/122 2023-06-23T01:09:29.887 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 15/122 2023-06-23T01:09:29.888 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 16/122 2023-06-23T01:09:29.888 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librados-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 17/122 2023-06-23T01:09:29.888 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_ 18/122 2023-06-23T01:09:29.888 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 19/122 2023-06-23T01:09:29.888 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-ceph-argparse-2:17.2.6-719.g42ddc46e.el8 20/122 2023-06-23T01:09:29.888 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-ceph-common-2:17.2.6-719.g42ddc46e.el8.x 21/122 2023-06-23T01:09:29.889 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cephfs-2:17.2.6-719.g42ddc46e.el8.x86_64 22/122 2023-06-23T01:09:29.889 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rados-2:17.2.6-719.g42ddc46e.el8.x86_64 23/122 2023-06-23T01:09:29.889 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rbd-2:17.2.6-719.g42ddc46e.el8.x86_64 24/122 2023-06-23T01:09:29.889 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rgw-2:17.2.6-719.g42ddc46e.el8.x86_64 25/122 2023-06-23T01:09:29.896 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : rbd-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 26/122 2023-06-23T01:09:29.896 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 27/122 2023-06-23T01:09:29.896 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : rbd-nbd-2:17.2.6-719.g42ddc46e.el8.x86_64 28/122 2023-06-23T01:09:29.897 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-719.g42ddc46e.e 29/122 2023-06-23T01:09:29.897 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noar 30/122 2023-06-23T01:09:29.897 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.no 31/122 2023-06-23T01:09:29.897 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-719.g42dd 32/122 2023-06-23T01:09:29.897 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-719.g42ddc46e.el8 33/122 2023-06-23T01:09:29.897 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 34/122 2023-06-23T01:09:29.897 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-719.g42ddc46e.el 35/122 2023-06-23T01:09:29.898 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 36/122 2023-06-23T01:09:29.898 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 37/122 2023-06-23T01:09:29.898 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephfs-top-2:17.2.6-719.g42ddc46e.el8.noarch 38/122 2023-06-23T01:09:29.898 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-06-23T01:09:29.898 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-06-23T01:09:29.898 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-06-23T01:09:29.898 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-06-23T01:09:29.899 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-06-23T01:09:29.899 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-06-23T01:09:29.899 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-06-23T01:09:29.899 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-06-23T01:09:29.899 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-06-23T01:09:29.899 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-06-23T01:09:29.899 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-06-23T01:09:29.900 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-06-23T01:09:29.900 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-06-23T01:09:29.900 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-06-23T01:09:29.900 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-06-23T01:09:29.900 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-06-23T01:09:29.900 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-06-23T01:09:29.900 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-06-23T01:09:29.901 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-06-23T01:09:29.901 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-06-23T01:09:29.901 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-06-23T01:09:29.901 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/122 2023-06-23T01:09:29.901 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-06-23T01:09:29.901 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-06-23T01:09:29.901 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-06-23T01:09:29.902 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-06-23T01:09:29.902 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-06-23T01:09:29.902 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-06-23T01:09:29.902 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-06-23T01:09:29.902 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-06-23T01:09:29.902 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-06-23T01:09:29.902 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-06-23T01:09:29.903 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-06-23T01:09:29.903 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 72/122 2023-06-23T01:09:29.903 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/122 2023-06-23T01:09:29.903 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 74/122 2023-06-23T01:09:29.903 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : jq-1.5-12.el8.x86_64 75/122 2023-06-23T01:09:29.903 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flex-2.6.1-9.el8.x86_64 76/122 2023-06-23T01:09:29.903 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/122 2023-06-23T01:09:29.904 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 78/122 2023-06-23T01:09:29.904 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 79/122 2023-06-23T01:09:29.904 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : bison-3.0.4-10.el8.x86_64 80/122 2023-06-23T01:09:29.904 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 81/122 2023-06-23T01:09:29.904 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 82/122 2023-06-23T01:09:29.904 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 83/122 2023-06-23T01:09:29.904 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 84/122 2023-06-23T01:09:29.905 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 85/122 2023-06-23T01:09:29.905 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 86/122 2023-06-23T01:09:29.905 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 87/122 2023-06-23T01:09:29.905 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 88/122 2023-06-23T01:09:29.905 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 89/122 2023-06-23T01:09:29.905 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 90/122 2023-06-23T01:09:29.905 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 91/122 2023-06-23T01:09:29.906 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 92/122 2023-06-23T01:09:29.906 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 93/122 2023-06-23T01:09:29.906 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 94/122 2023-06-23T01:09:29.906 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 95/122 2023-06-23T01:09:29.906 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : automake-1.16.1-7.el8.noarch 96/122 2023-06-23T01:09:29.906 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 97/122 2023-06-23T01:09:29.906 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 98/122 2023-06-23T01:09:29.907 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 99/122 2023-06-23T01:09:29.907 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 100/122 2023-06-23T01:09:29.907 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 101/122 2023-06-23T01:09:29.907 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 102/122 2023-06-23T01:09:29.907 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 103/122 2023-06-23T01:09:29.907 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 104/122 2023-06-23T01:09:29.908 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 105/122 2023-06-23T01:09:29.908 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 106/122 2023-06-23T01:09:29.908 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 107/122 2023-06-23T01:09:29.908 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 108/122 2023-06-23T01:09:29.908 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 109/122 2023-06-23T01:09:29.908 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 110/122 2023-06-23T01:09:29.908 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 111/122 2023-06-23T01:09:29.909 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 112/122 2023-06-23T01:09:29.909 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 113/122 2023-06-23T01:09:29.909 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 114/122 2023-06-23T01:09:29.909 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 115/122 2023-06-23T01:09:29.909 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 116/122 2023-06-23T01:09:29.909 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 117/122 2023-06-23T01:09:29.909 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 118/122 2023-06-23T01:09:29.910 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 119/122 2023-06-23T01:09:29.910 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-06-23T01:09:29.910 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 121/122 2023-06-23T01:09:32.232 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-06-23T01:09:32.233 INFO:teuthology.orchestra.run.smithi120.stdout:Last metadata expiration check: 0:01:53 ago on Fri 23 Jun 2023 01:07:39 AM UTC. 2023-06-23T01:09:34.317 INFO:teuthology.orchestra.run.smithi120.stdout:Installed products updated. 2023-06-23T01:09:34.447 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:09:34.447 INFO:teuthology.orchestra.run.smithi120.stdout:Upgraded: 2023-06-23T01:09:34.448 INFO:teuthology.orchestra.run.smithi120.stdout: librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:34.448 INFO:teuthology.orchestra.run.smithi120.stdout: librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:34.448 INFO:teuthology.orchestra.run.smithi120.stdout:Installed: 2023-06-23T01:09:34.449 INFO:teuthology.orchestra.run.smithi120.stdout: automake-1.16.1-7.el8.noarch 2023-06-23T01:09:34.449 INFO:teuthology.orchestra.run.smithi120.stdout: bison-3.0.4-10.el8.x86_64 2023-06-23T01:09:34.449 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:34.449 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:34.449 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:34.450 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:34.450 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-grafana-dashboards-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:09:34.450 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-immutable-object-cache-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:34.450 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:34.450 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:34.450 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:09:34.450 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:09:34.451 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-diskprediction-local-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:09:34.451 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-modules-core-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:09:34.451 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:09:34.451 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:34.451 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:34.452 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-prometheus-alerts-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:09:34.452 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:34.453 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:34.453 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-test-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:34.453 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:09:34.454 INFO:teuthology.orchestra.run.smithi120.stdout: cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:09:34.454 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:34.454 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-top-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:09:34.454 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-06-23T01:09:34.454 INFO:teuthology.orchestra.run.smithi120.stdout: flex-2.6.1-9.el8.x86_64 2023-06-23T01:09:34.455 INFO:teuthology.orchestra.run.smithi120.stdout: fuse-2.9.7-12.el8.x86_64 2023-06-23T01:09:34.455 INFO:teuthology.orchestra.run.smithi120.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-06-23T01:09:34.455 INFO:teuthology.orchestra.run.smithi120.stdout: jq-1.5-12.el8.x86_64 2023-06-23T01:09:34.455 INFO:teuthology.orchestra.run.smithi120.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-06-23T01:09:34.455 INFO:teuthology.orchestra.run.smithi120.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-06-23T01:09:34.455 INFO:teuthology.orchestra.run.smithi120.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-06-23T01:09:34.455 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:34.455 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:34.456 INFO:teuthology.orchestra.run.smithi120.stdout: libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:34.456 INFO:teuthology.orchestra.run.smithi120.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-06-23T01:09:34.456 INFO:teuthology.orchestra.run.smithi120.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-06-23T01:09:34.457 INFO:teuthology.orchestra.run.smithi120.stdout: liboath-2.6.2-3.el8.x86_64 2023-06-23T01:09:34.457 INFO:teuthology.orchestra.run.smithi120.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-06-23T01:09:34.457 INFO:teuthology.orchestra.run.smithi120.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-06-23T01:09:34.457 INFO:teuthology.orchestra.run.smithi120.stdout: librados-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:34.458 INFO:teuthology.orchestra.run.smithi120.stdout: libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:34.458 INFO:teuthology.orchestra.run.smithi120.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-06-23T01:09:34.458 INFO:teuthology.orchestra.run.smithi120.stdout: librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:34.458 INFO:teuthology.orchestra.run.smithi120.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-06-23T01:09:34.458 INFO:teuthology.orchestra.run.smithi120.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-06-23T01:09:34.458 INFO:teuthology.orchestra.run.smithi120.stdout: libtool-2.4.6-25.el8.x86_64 2023-06-23T01:09:34.458 INFO:teuthology.orchestra.run.smithi120.stdout: libunwind-1.3.1-3.el8.x86_64 2023-06-23T01:09:34.459 INFO:teuthology.orchestra.run.smithi120.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-06-23T01:09:34.459 INFO:teuthology.orchestra.run.smithi120.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-06-23T01:09:34.459 INFO:teuthology.orchestra.run.smithi120.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-06-23T01:09:34.459 INFO:teuthology.orchestra.run.smithi120.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-06-23T01:09:34.459 INFO:teuthology.orchestra.run.smithi120.stdout: openblas-0.3.12-1.el8.x86_64 2023-06-23T01:09:34.459 INFO:teuthology.orchestra.run.smithi120.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-06-23T01:09:34.459 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-06-23T01:09:34.459 INFO:teuthology.orchestra.run.smithi120.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-06-23T01:09:34.460 INFO:teuthology.orchestra.run.smithi120.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-06-23T01:09:34.460 INFO:teuthology.orchestra.run.smithi120.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-06-23T01:09:34.460 INFO:teuthology.orchestra.run.smithi120.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-06-23T01:09:34.460 INFO:teuthology.orchestra.run.smithi120.stdout: python3-babel-2.5.1-5.el8.noarch 2023-06-23T01:09:34.460 INFO:teuthology.orchestra.run.smithi120.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-06-23T01:09:34.460 INFO:teuthology.orchestra.run.smithi120.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-06-23T01:09:34.460 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-06-23T01:09:34.461 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-argparse-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:34.461 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:34.461 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cephfs-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:34.461 INFO:teuthology.orchestra.run.smithi120.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-06-23T01:09:34.461 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-06-23T01:09:34.461 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-06-23T01:09:34.461 INFO:teuthology.orchestra.run.smithi120.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-06-23T01:09:34.461 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-6.2-6.el8.noarch 2023-06-23T01:09:34.463 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-06-23T01:09:34.463 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-06-23T01:09:34.463 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-06-23T01:09:34.463 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-06-23T01:09:34.463 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-06-23T01:09:34.464 INFO:teuthology.orchestra.run.smithi120.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-06-23T01:09:34.464 INFO:teuthology.orchestra.run.smithi120.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-06-23T01:09:34.464 INFO:teuthology.orchestra.run.smithi120.stdout: python3-mako-1.0.6-13.el8.noarch 2023-06-23T01:09:34.464 INFO:teuthology.orchestra.run.smithi120.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-06-23T01:09:34.464 INFO:teuthology.orchestra.run.smithi120.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-06-23T01:09:34.464 INFO:teuthology.orchestra.run.smithi120.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-06-23T01:09:34.464 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-06-23T01:09:34.465 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-06-23T01:09:34.465 INFO:teuthology.orchestra.run.smithi120.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-06-23T01:09:34.465 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-06-23T01:09:34.465 INFO:teuthology.orchestra.run.smithi120.stdout: python3-portend-2.6-1.el8.noarch 2023-06-23T01:09:34.465 INFO:teuthology.orchestra.run.smithi120.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-06-23T01:09:34.465 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-06-23T01:09:34.465 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-06-23T01:09:34.466 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pytz-2017.2-9.el8.noarch 2023-06-23T01:09:34.466 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rados-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:34.466 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rbd-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:34.466 INFO:teuthology.orchestra.run.smithi120.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-06-23T01:09:34.466 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-06-23T01:09:34.466 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rgw-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:34.466 INFO:teuthology.orchestra.run.smithi120.stdout: python3-routes-2.4.1-12.el8.noarch 2023-06-23T01:09:34.466 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rsa-4.8-1.el8.noarch 2023-06-23T01:09:34.467 INFO:teuthology.orchestra.run.smithi120.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-06-23T01:09:34.467 INFO:teuthology.orchestra.run.smithi120.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-06-23T01:09:34.467 INFO:teuthology.orchestra.run.smithi120.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-06-23T01:09:34.467 INFO:teuthology.orchestra.run.smithi120.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-06-23T01:09:34.467 INFO:teuthology.orchestra.run.smithi120.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-06-23T01:09:34.467 INFO:teuthology.orchestra.run.smithi120.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-06-23T01:09:34.467 INFO:teuthology.orchestra.run.smithi120.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-06-23T01:09:34.467 INFO:teuthology.orchestra.run.smithi120.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-06-23T01:09:34.468 INFO:teuthology.orchestra.run.smithi120.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-06-23T01:09:34.468 INFO:teuthology.orchestra.run.smithi120.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-06-23T01:09:34.468 INFO:teuthology.orchestra.run.smithi120.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-06-23T01:09:34.468 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:34.468 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:34.468 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-nbd-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:09:34.468 INFO:teuthology.orchestra.run.smithi120.stdout: socat-1.7.3.3-2.el8.x86_64 2023-06-23T01:09:34.469 INFO:teuthology.orchestra.run.smithi120.stdout: thrift-0.13.0-2.el8.x86_64 2023-06-23T01:09:34.469 INFO:teuthology.orchestra.run.smithi120.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-06-23T01:09:34.469 INFO:teuthology.orchestra.run.smithi120.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-06-23T01:09:34.469 INFO:teuthology.orchestra.run.smithi120.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-06-23T01:09:34.470 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:09:34.470 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2023-06-23T01:09:34.760 DEBUG:teuthology.parallel:result is None 2023-06-23T01:09:34.760 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=42ddc46e6e6582d8e29a5a982a81886fe028be2c 2023-06-23T01:09:34.978 DEBUG:teuthology.orchestra.run.smithi120:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-06-23T01:09:35.030 INFO:teuthology.orchestra.run.smithi120.stdout:17.2.6-719.g42ddc46e.el8 2023-06-23T01:09:35.030 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-719.g42ddc46e.el8 2023-06-23T01:09:35.031 INFO:teuthology.task.install:The correct ceph version 17.2.6-719.g42ddc46e is installed. 2023-06-23T01:09:35.031 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=42ddc46e6e6582d8e29a5a982a81886fe028be2c 2023-06-23T01:09:35.175 DEBUG:teuthology.orchestra.run.smithi134:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-06-23T01:09:35.201 INFO:teuthology.orchestra.run.smithi134.stdout:17.2.6-719.g42ddc46e.el8 2023-06-23T01:09:35.201 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-719.g42ddc46e.el8 2023-06-23T01:09:35.202 INFO:teuthology.task.install:The correct ceph version 17.2.6-719.g42ddc46e is installed. 2023-06-23T01:09:35.203 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=42ddc46e6e6582d8e29a5a982a81886fe028be2c 2023-06-23T01:09:35.341 DEBUG:teuthology.orchestra.run.smithi157:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-06-23T01:09:35.366 INFO:teuthology.orchestra.run.smithi157.stdout:17.2.6-719.g42ddc46e.el8 2023-06-23T01:09:35.366 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-719.g42ddc46e.el8 2023-06-23T01:09:35.367 INFO:teuthology.task.install:The correct ceph version 17.2.6-719.g42ddc46e is installed. 2023-06-23T01:09:35.367 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-06-23T01:09:35.368 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2023-06-23T01:09:35.368 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-06-23T01:09:35.448 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-06-23T01:09:35.448 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-06-23T01:09:35.491 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-06-23T01:09:35.492 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-06-23T01:09:35.534 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-06-23T01:09:35.534 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2023-06-23T01:09:35.535 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/daemon-helper 2023-06-23T01:09:35.595 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-06-23T01:09:35.694 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-06-23T01:09:35.695 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/daemon-helper 2023-06-23T01:09:35.724 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-06-23T01:09:35.791 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-06-23T01:09:35.791 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/daemon-helper 2023-06-23T01:09:35.820 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-06-23T01:09:35.888 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-06-23T01:09:35.889 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2023-06-23T01:09:35.889 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/adjust-ulimits 2023-06-23T01:09:35.949 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-06-23T01:09:36.048 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-06-23T01:09:36.048 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/adjust-ulimits 2023-06-23T01:09:36.076 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-06-23T01:09:36.143 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-06-23T01:09:36.143 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/adjust-ulimits 2023-06-23T01:09:36.171 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-06-23T01:09:36.239 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-06-23T01:09:36.239 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2023-06-23T01:09:36.239 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/stdin-killer 2023-06-23T01:09:36.299 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-06-23T01:09:36.401 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-06-23T01:09:36.401 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/stdin-killer 2023-06-23T01:09:36.429 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-06-23T01:09:36.497 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-06-23T01:09:36.498 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/stdin-killer 2023-06-23T01:09:36.525 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-06-23T01:09:36.592 INFO:teuthology.run_tasks:Running task cephadm... 2023-06-23T01:09:36.674 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2023-06-23T01:09:36.675 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c 2023-06-23T01:09:36.684 INFO:tasks.cephadm:Cluster fsid is 9efbfa8e-1162-11ee-9b2b-001a4aab830c 2023-06-23T01:09:36.685 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-06-23T01:09:36.685 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.120', 'mon.b': '172.21.15.134', 'mon.c': '172.21.15.157'} 2023-06-23T01:09:36.686 INFO:tasks.cephadm:First mon is mon.a on smithi120 2023-06-23T01:09:36.686 INFO:tasks.cephadm:First mgr is x 2023-06-23T01:09:36.686 INFO:tasks.cephadm:Normalizing hostnames... 2023-06-23T01:09:36.687 DEBUG:teuthology.orchestra.run.smithi120:> sudo hostname $(hostname -s) 2023-06-23T01:09:36.749 DEBUG:teuthology.orchestra.run.smithi134:> sudo hostname $(hostname -s) 2023-06-23T01:09:36.777 DEBUG:teuthology.orchestra.run.smithi157:> sudo hostname $(hostname -s) 2023-06-23T01:09:36.807 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph-ci.git ref 42ddc46e6e6582d8e29a5a982a81886fe028be2c)... 2023-06-23T01:09:36.807 DEBUG:teuthology.orchestra.run.smithi120:> curl --silent https://raw.githubusercontent.com/ceph/ceph-ci/42ddc46e6e6582d8e29a5a982a81886fe028be2c/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-06-23T01:09:37.133 INFO:teuthology.orchestra.run.smithi120.stdout:-rw-rw-r--. 1 ubuntu ubuntu 366215 Jun 23 01:09 /home/ubuntu/cephtest/cephadm 2023-06-23T01:09:37.133 DEBUG:teuthology.orchestra.run.smithi134:> curl --silent https://raw.githubusercontent.com/ceph/ceph-ci/42ddc46e6e6582d8e29a5a982a81886fe028be2c/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-06-23T01:09:37.294 INFO:teuthology.orchestra.run.smithi134.stdout:-rw-rw-r--. 1 ubuntu ubuntu 366215 Jun 23 01:09 /home/ubuntu/cephtest/cephadm 2023-06-23T01:09:37.294 DEBUG:teuthology.orchestra.run.smithi157:> curl --silent https://raw.githubusercontent.com/ceph/ceph-ci/42ddc46e6e6582d8e29a5a982a81886fe028be2c/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-06-23T01:09:37.478 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-rw-r--. 1 ubuntu ubuntu 366215 Jun 23 01:09 /home/ubuntu/cephtest/cephadm 2023-06-23T01:09:37.479 DEBUG:teuthology.orchestra.run.smithi120:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-06-23T01:09:37.519 DEBUG:teuthology.orchestra.run.smithi134:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-06-23T01:09:37.537 DEBUG:teuthology.orchestra.run.smithi157:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-06-23T01:09:37.570 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c on all hosts... 2023-06-23T01:09:37.570 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c pull 2023-06-23T01:09:37.575 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c pull 2023-06-23T01:09:37.579 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c pull 2023-06-23T01:09:42.808 INFO:teuthology.orchestra.run.smithi157.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c... 2023-06-23T01:09:43.031 INFO:teuthology.orchestra.run.smithi134.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c... 2023-06-23T01:09:43.260 INFO:teuthology.orchestra.run.smithi120.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c... 2023-06-23T01:10:18.706 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2023-06-23T01:10:18.706 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_version": "ceph version 17.2.6-719-g42ddc46e (42ddc46e6e6582d8e29a5a982a81886fe028be2c) quincy (stable)", 2023-06-23T01:10:18.707 INFO:teuthology.orchestra.run.smithi120.stdout: "image_id": "f3276efde470d44ef0ff8e259464d1d34a5015beb9d7c1215520bcf3b48fd710", 2023-06-23T01:10:18.707 INFO:teuthology.orchestra.run.smithi120.stdout: "repo_digests": [ 2023-06-23T01:10:18.707 INFO:teuthology.orchestra.run.smithi120.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b9658f1f8fdc87272ce6f577d5ecedf811fa77b9a03555eb6b1de17c25ce4236" 2023-06-23T01:10:18.707 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2023-06-23T01:10:18.707 INFO:teuthology.orchestra.run.smithi120.stdout:} 2023-06-23T01:10:23.911 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2023-06-23T01:10:23.911 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_version": "ceph version 17.2.6-719-g42ddc46e (42ddc46e6e6582d8e29a5a982a81886fe028be2c) quincy (stable)", 2023-06-23T01:10:23.912 INFO:teuthology.orchestra.run.smithi134.stdout: "image_id": "f3276efde470d44ef0ff8e259464d1d34a5015beb9d7c1215520bcf3b48fd710", 2023-06-23T01:10:23.912 INFO:teuthology.orchestra.run.smithi134.stdout: "repo_digests": [ 2023-06-23T01:10:23.912 INFO:teuthology.orchestra.run.smithi134.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b9658f1f8fdc87272ce6f577d5ecedf811fa77b9a03555eb6b1de17c25ce4236" 2023-06-23T01:10:23.912 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2023-06-23T01:10:23.913 INFO:teuthology.orchestra.run.smithi134.stdout:} 2023-06-23T01:10:29.184 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-06-23T01:10:29.185 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 17.2.6-719-g42ddc46e (42ddc46e6e6582d8e29a5a982a81886fe028be2c) quincy (stable)", 2023-06-23T01:10:29.185 INFO:teuthology.orchestra.run.smithi157.stdout: "image_id": "f3276efde470d44ef0ff8e259464d1d34a5015beb9d7c1215520bcf3b48fd710", 2023-06-23T01:10:29.185 INFO:teuthology.orchestra.run.smithi157.stdout: "repo_digests": [ 2023-06-23T01:10:29.185 INFO:teuthology.orchestra.run.smithi157.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b9658f1f8fdc87272ce6f577d5ecedf811fa77b9a03555eb6b1de17c25ce4236" 2023-06-23T01:10:29.185 INFO:teuthology.orchestra.run.smithi157.stdout: ] 2023-06-23T01:10:29.185 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-06-23T01:10:29.214 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /etc/ceph 2023-06-23T01:10:29.286 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /etc/ceph 2023-06-23T01:10:29.320 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /etc/ceph 2023-06-23T01:10:29.354 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 777 /etc/ceph 2023-06-23T01:10:29.417 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 777 /etc/ceph 2023-06-23T01:10:29.445 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 777 /etc/ceph 2023-06-23T01:10:29.474 INFO:tasks.cephadm:Writing seed config... 2023-06-23T01:10:29.476 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-06-23T01:10:29.476 INFO:tasks.cephadm: override: [client] debug client = 20 2023-06-23T01:10:29.476 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-06-23T01:10:29.476 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-06-23T01:10:29.477 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-06-23T01:10:29.477 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-06-23T01:10:29.477 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-06-23T01:10:29.477 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-06-23T01:10:29.477 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-06-23T01:10:29.478 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-06-23T01:10:29.478 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-06-23T01:10:29.478 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-06-23T01:10:29.478 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-06-23T01:10:29.478 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-06-23T01:10:29.479 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-06-23T01:10:29.479 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-06-23T01:10:29.479 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-06-23T01:10:29.479 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-06-23T01:10:29.479 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-06-23T01:10:29.479 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-06-23T01:10:29.480 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-06-23T01:10:29.480 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-06-23T01:10:29.480 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-06-23T01:10:29.480 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-06-23T01:10:29.480 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-06-23T01:10:29.481 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2023-06-23T01:10:29.481 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2023-06-23T01:10:29.481 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2023-06-23T01:10:29.481 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-06-23T01:10:29.481 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-06-23T01:10:29.482 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2023-06-23T01:10:29.482 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2023-06-23T01:10:29.482 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-06-23T01:10:29.482 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-06-23T01:10:29.482 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2023-06-23T01:10:29.483 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-06-23T01:10:29.483 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-06-23T01:10:29.483 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-06-23T01:10:29.483 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-06-23T01:10:29.483 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-06-23T01:10:29.484 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-06-23T01:10:29.484 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-06-23T01:10:29.484 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2023-06-23T01:10:29.485 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2023-06-23T01:10:29.485 DEBUG:teuthology.orchestra.run.smithi120:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-06-23T01:10:29.521 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 9efbfa8e-1162-11ee-9b2b-001a4aab830c ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-06-23T01:10:29.522 DEBUG:teuthology.orchestra.run.smithi120:mon.a> sudo journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mon.a.service 2023-06-23T01:10:29.566 DEBUG:teuthology.orchestra.run.smithi120:mgr.x> sudo journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mgr.x.service 2023-06-23T01:10:29.610 INFO:tasks.cephadm:Bootstrapping... 2023-06-23T01:10:29.610 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c -v bootstrap --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.120 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-06-23T01:10:29.624 INFO:journalctl@ceph.mon.a.smithi120.stdout:-- Logs begin at Fri 2023-06-23 00:53:53 UTC. -- 2023-06-23T01:10:29.672 INFO:journalctl@ceph.mgr.x.smithi120.stdout:-- Logs begin at Fri 2023-06-23 00:53:53 UTC. -- 2023-06-23T01:10:29.871 INFO:teuthology.orchestra.run.smithi120.stdout:-------------------------------------------------------------------------------- 2023-06-23T01:10:29.871 INFO:teuthology.orchestra.run.smithi120.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c', '-v', 'bootstrap', '--fsid', '9efbfa8e-1162-11ee-9b2b-001a4aab830c', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.120', '--skip-admin-label'] 2023-06-23T01:10:30.039 INFO:teuthology.orchestra.run.smithi120.stdout:/bin/podman: stdout 3.2.3 2023-06-23T01:10:30.040 INFO:teuthology.orchestra.run.smithi120.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-06-23T01:10:30.040 INFO:teuthology.orchestra.run.smithi120.stdout:Verifying podman|docker is present... 2023-06-23T01:10:30.207 INFO:teuthology.orchestra.run.smithi120.stdout:/bin/podman: stdout 3.2.3 2023-06-23T01:10:30.208 INFO:teuthology.orchestra.run.smithi120.stdout:Verifying lvm2 is present... 2023-06-23T01:10:30.208 INFO:teuthology.orchestra.run.smithi120.stdout:Verifying time synchronization is in place... 2023-06-23T01:10:30.221 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-06-23T01:10:30.222 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-06-23T01:10:30.234 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-06-23T01:10:30.234 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout inactive 2023-06-23T01:10:30.248 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout enabled 2023-06-23T01:10:30.260 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout active 2023-06-23T01:10:30.261 INFO:teuthology.orchestra.run.smithi120.stdout:Unit chronyd.service is enabled and running 2023-06-23T01:10:30.261 INFO:teuthology.orchestra.run.smithi120.stdout:Repeating the final host check... 2023-06-23T01:10:30.428 INFO:teuthology.orchestra.run.smithi120.stdout:/bin/podman: stdout 3.2.3 2023-06-23T01:10:30.428 INFO:teuthology.orchestra.run.smithi120.stdout:podman (/bin/podman) version 3.2.3 is present 2023-06-23T01:10:30.428 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl is present 2023-06-23T01:10:30.428 INFO:teuthology.orchestra.run.smithi120.stdout:lvcreate is present 2023-06-23T01:10:30.441 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-06-23T01:10:30.441 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-06-23T01:10:30.453 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-06-23T01:10:30.453 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout inactive 2023-06-23T01:10:30.467 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout enabled 2023-06-23T01:10:30.479 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout active 2023-06-23T01:10:30.480 INFO:teuthology.orchestra.run.smithi120.stdout:Unit chronyd.service is enabled and running 2023-06-23T01:10:30.480 INFO:teuthology.orchestra.run.smithi120.stdout:Host looks OK 2023-06-23T01:10:30.480 INFO:teuthology.orchestra.run.smithi120.stdout:Cluster fsid: 9efbfa8e-1162-11ee-9b2b-001a4aab830c 2023-06-23T01:10:30.481 INFO:teuthology.orchestra.run.smithi120.stdout:Acquiring lock 139766106823144 on /run/cephadm/9efbfa8e-1162-11ee-9b2b-001a4aab830c.lock 2023-06-23T01:10:30.481 INFO:teuthology.orchestra.run.smithi120.stdout:Lock 139766106823144 acquired on /run/cephadm/9efbfa8e-1162-11ee-9b2b-001a4aab830c.lock 2023-06-23T01:10:30.481 INFO:teuthology.orchestra.run.smithi120.stdout:Verifying IP 172.21.15.120 port 3300 ... 2023-06-23T01:10:30.481 INFO:teuthology.orchestra.run.smithi120.stdout:Verifying IP 172.21.15.120 port 6789 ... 2023-06-23T01:10:30.481 INFO:teuthology.orchestra.run.smithi120.stdout:Base mon IP(s) is [172.21.15.120:3300, 172.21.15.120:6789], mon addrv is [v2:172.21.15.120:3300,v1:172.21.15.120:6789] 2023-06-23T01:10:30.488 INFO:teuthology.orchestra.run.smithi120.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp metric 100 2023-06-23T01:10:30.488 INFO:teuthology.orchestra.run.smithi120.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.120 metric 100 2023-06-23T01:10:30.494 INFO:teuthology.orchestra.run.smithi120.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-06-23T01:10:30.494 INFO:teuthology.orchestra.run.smithi120.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2023-06-23T01:10:30.494 INFO:teuthology.orchestra.run.smithi120.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1789sec hoplimit 64 pref medium 2023-06-23T01:10:30.500 INFO:teuthology.orchestra.run.smithi120.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-06-23T01:10:30.500 INFO:teuthology.orchestra.run.smithi120.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-06-23T01:10:30.501 INFO:teuthology.orchestra.run.smithi120.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-06-23T01:10:30.501 INFO:teuthology.orchestra.run.smithi120.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2023-06-23T01:10:30.501 INFO:teuthology.orchestra.run.smithi120.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:478f/64 scope link 2023-06-23T01:10:30.501 INFO:teuthology.orchestra.run.smithi120.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-06-23T01:10:30.502 INFO:teuthology.orchestra.run.smithi120.stdout:Mon IP `172.21.15.120` is in CIDR network `172.21.0.0/20` 2023-06-23T01:10:30.502 INFO:teuthology.orchestra.run.smithi120.stdout:Mon IP `172.21.15.120` is in CIDR network `172.21.0.0/20` 2023-06-23T01:10:30.502 INFO:teuthology.orchestra.run.smithi120.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-06-23T01:10:30.502 INFO:teuthology.orchestra.run.smithi120.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-06-23T01:10:30.503 INFO:teuthology.orchestra.run.smithi120.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c... 2023-06-23T01:10:30.971 INFO:teuthology.orchestra.run.smithi120.stdout:/bin/podman: stdout f3276efde470d44ef0ff8e259464d1d34a5015beb9d7c1215520bcf3b48fd710 2023-06-23T01:10:30.972 INFO:teuthology.orchestra.run.smithi120.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c... 2023-06-23T01:10:30.972 INFO:teuthology.orchestra.run.smithi120.stdout:/bin/podman: stderr Getting image source signatures 2023-06-23T01:10:30.972 INFO:teuthology.orchestra.run.smithi120.stdout:/bin/podman: stderr Copying blob sha256:9d60a6994110d194fea96cb2ff67b650891faff48e39f8a6b8ad8152d347e90d 2023-06-23T01:10:30.972 INFO:teuthology.orchestra.run.smithi120.stdout:/bin/podman: stderr Copying blob sha256:3dbfccf0662dd88e0c2efa9d59916ea5c98f7c5a05bbe18e86c2dc32f0494158 2023-06-23T01:10:30.972 INFO:teuthology.orchestra.run.smithi120.stdout:/bin/podman: stderr Copying config sha256:f3276efde470d44ef0ff8e259464d1d34a5015beb9d7c1215520bcf3b48fd710 2023-06-23T01:10:30.972 INFO:teuthology.orchestra.run.smithi120.stdout:/bin/podman: stderr Writing manifest to image destination 2023-06-23T01:10:30.973 INFO:teuthology.orchestra.run.smithi120.stdout:/bin/podman: stderr Storing signatures 2023-06-23T01:10:31.841 INFO:teuthology.orchestra.run.smithi120.stdout:ceph: stdout ceph version 17.2.6-719-g42ddc46e (42ddc46e6e6582d8e29a5a982a81886fe028be2c) quincy (stable) 2023-06-23T01:10:31.842 INFO:teuthology.orchestra.run.smithi120.stdout:Ceph version: ceph version 17.2.6-719-g42ddc46e (42ddc46e6e6582d8e29a5a982a81886fe028be2c) quincy (stable) 2023-06-23T01:10:31.842 INFO:teuthology.orchestra.run.smithi120.stdout:Extracting ceph user uid/gid from container image... 2023-06-23T01:10:32.612 INFO:teuthology.orchestra.run.smithi120.stdout:stat: stdout 167 167 2023-06-23T01:10:32.612 INFO:teuthology.orchestra.run.smithi120.stdout:Creating initial keys... 2023-06-23T01:10:33.387 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph-authtool: stdout AQAJ8ZRk8RapABAAa+nQfin7pQwktUt5KJ51eQ== 2023-06-23T01:10:34.160 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph-authtool: stdout AQAJ8ZRkJ6rWLRAA7qV9e21/RoEUA7xVHk1zng== 2023-06-23T01:10:34.929 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph-authtool: stdout AQAK8ZRkb4bzHxAA8xaY4pWeKunPLXnlwvsd7g== 2023-06-23T01:10:34.931 INFO:teuthology.orchestra.run.smithi120.stdout:Creating initial monmap... 2023-06-23T01:10:35.700 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-06-23T01:10:35.700 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2023-06-23T01:10:35.700 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 9efbfa8e-1162-11ee-9b2b-001a4aab830c 2023-06-23T01:10:35.700 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-06-23T01:10:35.701 INFO:teuthology.orchestra.run.smithi120.stdout:monmaptool for a [v2:172.21.15.120:3300,v1:172.21.15.120:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-06-23T01:10:35.701 INFO:teuthology.orchestra.run.smithi120.stdout:setting min_mon_release = octopus 2023-06-23T01:10:35.701 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/monmaptool: set fsid to 9efbfa8e-1162-11ee-9b2b-001a4aab830c 2023-06-23T01:10:35.701 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-06-23T01:10:35.701 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:10:35.701 INFO:teuthology.orchestra.run.smithi120.stdout:Creating mon... 2023-06-23T01:10:36.468 INFO:teuthology.orchestra.run.smithi120.stdout:create mon.a on 2023-06-23T01:10:36.686 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-06-23T01:10:36.904 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-06-23T01:10:37.138 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c.target → /etc/systemd/system/ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c.target. 2023-06-23T01:10:37.139 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c.target → /etc/systemd/system/ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c.target. 2023-06-23T01:10:37.476 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mon.a 2023-06-23T01:10:37.476 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mon.a.service: Unit ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mon.a.service not loaded. 2023-06-23T01:10:37.725 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c.target.wants/ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mon.a.service → /etc/systemd/system/ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@.service. 2023-06-23T01:10:38.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:37 smithi120 systemd[1]: Starting Ceph mon.a for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:10:46.940 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-06-23T01:10:46.941 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout disabled 2023-06-23T01:10:46.958 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-06-23T01:10:46.959 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout inactive 2023-06-23T01:10:46.959 INFO:teuthology.orchestra.run.smithi120.stdout:firewalld.service is not enabled 2023-06-23T01:10:46.959 INFO:teuthology.orchestra.run.smithi120.stdout:Not possible to enable service . firewalld.service is not available 2023-06-23T01:10:46.960 INFO:teuthology.orchestra.run.smithi120.stdout:Waiting for mon to start... 2023-06-23T01:10:46.961 INFO:teuthology.orchestra.run.smithi120.stdout:Waiting for mon... 2023-06-23T01:10:47.231 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:46 smithi120 bash[78838]: aa891b1000550006d15a6e083b4aa73308a3b90daaf76f66a67c3cc445cc27fb 2023-06-23T01:10:47.231 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:46 smithi120 ceph-mon[78902]: mkfs 9efbfa8e-1162-11ee-9b2b-001a4aab830c 2023-06-23T01:10:47.231 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:46 smithi120 ceph-mon[78902]: mon.a is new leader, mons a in quorum (ranks 0) 2023-06-23T01:10:47.231 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:46 smithi120 systemd[1]: Started Ceph mon.a for 9efbfa8e-1162-11ee-9b2b-001a4aab830c. 2023-06-23T01:10:47.902 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:47 smithi120 ceph-mon[78902]: mon.a is new leader, mons a in quorum (ranks 0) 2023-06-23T01:10:47.903 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:47 smithi120 ceph-mon[78902]: monmap e1: 1 mons at {a=[v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0]} removed_ranks: {} 2023-06-23T01:10:47.903 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:47 smithi120 ceph-mon[78902]: fsmap 2023-06-23T01:10:47.903 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:47 smithi120 ceph-mon[78902]: osdmap e1: 0 total, 0 up, 0 in 2023-06-23T01:10:47.903 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:47 smithi120 ceph-mon[78902]: mgrmap e1: no daemons active 2023-06-23T01:10:47.904 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:47 smithi120 ceph-mon[78902]: from='client.? 172.21.15.120:0/381510708' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-06-23T01:10:47.937 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout cluster: 2023-06-23T01:10:47.938 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout id: 9efbfa8e-1162-11ee-9b2b-001a4aab830c 2023-06-23T01:10:47.938 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-06-23T01:10:47.938 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 2023-06-23T01:10:47.938 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout services: 2023-06-23T01:10:47.939 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.731667s) 2023-06-23T01:10:47.939 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-06-23T01:10:47.939 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-06-23T01:10:47.939 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 2023-06-23T01:10:47.939 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout data: 2023-06-23T01:10:47.939 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-06-23T01:10:47.939 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-06-23T01:10:47.939 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-06-23T01:10:47.940 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout pgs: 2023-06-23T01:10:47.940 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 2023-06-23T01:10:47.940 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.486+0000 7ff2e9ce6700 1 Processor -- start 2023-06-23T01:10:47.940 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.486+0000 7ff2e9ce6700 1 -- start start 2023-06-23T01:10:47.940 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.487+0000 7ff2e9ce6700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2e4104490 0x7ff2e41048a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:10:47.940 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.487+0000 7ff2e9ce6700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e4104de0 con 0x7ff2e4104490 2023-06-23T01:10:47.941 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.488+0000 7ff2e37fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2e4104490 0x7ff2e41048a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:10:47.941 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.488+0000 7ff2e37fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2e4104490 0x7ff2e41048a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58940/0 (socket says 172.21.15.120:58940) 2023-06-23T01:10:47.941 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.488+0000 7ff2e37fe700 1 -- 172.21.15.120:0/2817787536 learned_addr learned my addr 172.21.15.120:0/2817787536 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:10:47.941 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.488+0000 7ff2e37fe700 1 -- 172.21.15.120:0/2817787536 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e4104f20 con 0x7ff2e4104490 2023-06-23T01:10:47.941 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.488+0000 7ff2e37fe700 1 --2- 172.21.15.120:0/2817787536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2e4104490 0x7ff2e41048a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40098e0 tx=0x7ff2d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a380c68628d2a9a3 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:10:47.941 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.489+0000 7ff2e2ffd700 1 -- 172.21.15.120:0/2817787536 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff2d4009e80 con 0x7ff2e4104490 2023-06-23T01:10:47.941 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.489+0000 7ff2e2ffd700 1 -- 172.21.15.120:0/2817787536 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d400b860 con 0x7ff2e4104490 2023-06-23T01:10:47.941 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.489+0000 7ff2e2ffd700 1 -- 172.21.15.120:0/2817787536 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff2d400b9d0 con 0x7ff2e4104490 2023-06-23T01:10:47.942 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.489+0000 7ff2e9ce6700 1 -- 172.21.15.120:0/2817787536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2e4104490 msgr2=0x7ff2e41048a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:10:47.942 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.489+0000 7ff2e9ce6700 1 --2- 172.21.15.120:0/2817787536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2e4104490 0x7ff2e41048a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40098e0 tx=0x7ff2d4009250 comp rx=0 tx=0).stop 2023-06-23T01:10:47.942 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.489+0000 7ff2e9ce6700 1 -- 172.21.15.120:0/2817787536 shutdown_connections 2023-06-23T01:10:47.942 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.489+0000 7ff2e9ce6700 1 --2- 172.21.15.120:0/2817787536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2e4104490 0x7ff2e41048a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:10:47.942 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.489+0000 7ff2e9ce6700 1 -- 172.21.15.120:0/2817787536 >> 172.21.15.120:0/2817787536 conn(0x7ff2e4077a30 msgr2=0x7ff2e41023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:10:47.943 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.489+0000 7ff2e9ce6700 1 -- 172.21.15.120:0/2817787536 shutdown_connections 2023-06-23T01:10:47.943 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.489+0000 7ff2e9ce6700 1 -- 172.21.15.120:0/2817787536 wait complete. 2023-06-23T01:10:47.943 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.490+0000 7ff2e9ce6700 1 Processor -- start 2023-06-23T01:10:47.943 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.490+0000 7ff2e9ce6700 1 -- start start 2023-06-23T01:10:47.943 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.491+0000 7ff2e9ce6700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2e4104490 0x7ff2e4112910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:10:47.943 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.491+0000 7ff2e9ce6700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e4112e50 con 0x7ff2e4104490 2023-06-23T01:10:47.943 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.491+0000 7ff2e37fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2e4104490 0x7ff2e4112910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:10:47.944 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.491+0000 7ff2e37fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2e4104490 0x7ff2e4112910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58944/0 (socket says 172.21.15.120:58944) 2023-06-23T01:10:47.944 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.491+0000 7ff2e37fe700 1 -- 172.21.15.120:0/381510708 learned_addr learned my addr 172.21.15.120:0/381510708 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:10:47.944 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.491+0000 7ff2e37fe700 1 -- 172.21.15.120:0/381510708 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d40094f0 con 0x7ff2e4104490 2023-06-23T01:10:47.944 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.491+0000 7ff2e37fe700 1 --2- 172.21.15.120:0/381510708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2e4104490 0x7ff2e4112910 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4003080 tx=0x7ff2d4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:10:47.944 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.492+0000 7ff2e0ff9700 1 -- 172.21.15.120:0/381510708 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff2d401c460 con 0x7ff2e4104490 2023-06-23T01:10:47.944 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.492+0000 7ff2e0ff9700 1 -- 172.21.15.120:0/381510708 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d401c5c0 con 0x7ff2e4104490 2023-06-23T01:10:47.944 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.492+0000 7ff2e9ce6700 1 -- 172.21.15.120:0/381510708 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2e4113050 con 0x7ff2e4104490 2023-06-23T01:10:47.945 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.492+0000 7ff2e0ff9700 1 -- 172.21.15.120:0/381510708 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff2d401c890 con 0x7ff2e4104490 2023-06-23T01:10:47.945 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.492+0000 7ff2e9ce6700 1 -- 172.21.15.120:0/381510708 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2e4113cf0 con 0x7ff2e4104490 2023-06-23T01:10:47.945 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.492+0000 7ff2e0ff9700 1 -- 172.21.15.120:0/381510708 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7ff2d401a450 con 0x7ff2e4104490 2023-06-23T01:10:47.945 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.492+0000 7ff2e0ff9700 1 -- 172.21.15.120:0/381510708 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7ff2d402e790 con 0x7ff2e4104490 2023-06-23T01:10:47.945 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.493+0000 7ff2e9ce6700 1 -- 172.21.15.120:0/381510708 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2e405f060 con 0x7ff2e4104490 2023-06-23T01:10:47.945 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.496+0000 7ff2e0ff9700 1 -- 172.21.15.120:0/381510708 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7ff2d401cd80 con 0x7ff2e4104490 2023-06-23T01:10:47.945 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.569+0000 7ff2e9ce6700 1 -- 172.21.15.120:0/381510708 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7ff2e41147c0 con 0x7ff2e4104490 2023-06-23T01:10:47.946 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.569+0000 7ff2e0ff9700 1 -- 172.21.15.120:0/381510708 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7ff2d401a890 con 0x7ff2e4104490 2023-06-23T01:10:47.946 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.570+0000 7ff2e9ce6700 1 -- 172.21.15.120:0/381510708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2e4104490 msgr2=0x7ff2e4112910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:10:47.946 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.570+0000 7ff2e9ce6700 1 --2- 172.21.15.120:0/381510708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2e4104490 0x7ff2e4112910 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4003080 tx=0x7ff2d4016000 comp rx=0 tx=0).stop 2023-06-23T01:10:47.946 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.570+0000 7ff2e9ce6700 1 -- 172.21.15.120:0/381510708 shutdown_connections 2023-06-23T01:10:47.946 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.570+0000 7ff2e9ce6700 1 --2- 172.21.15.120:0/381510708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2e4104490 0x7ff2e4112910 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:10:47.946 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.570+0000 7ff2e9ce6700 1 -- 172.21.15.120:0/381510708 >> 172.21.15.120:0/381510708 conn(0x7ff2e4077a30 msgr2=0x7ff2e41071f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:10:47.946 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.570+0000 7ff2e9ce6700 1 -- 172.21.15.120:0/381510708 shutdown_connections 2023-06-23T01:10:47.946 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:47.570+0000 7ff2e9ce6700 1 -- 172.21.15.120:0/381510708 wait complete. 2023-06-23T01:10:47.947 INFO:teuthology.orchestra.run.smithi120.stdout:mon is available 2023-06-23T01:10:47.947 INFO:teuthology.orchestra.run.smithi120.stdout:Assimilating anything we can from ceph.conf... 2023-06-23T01:10:48.906 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 2023-06-23T01:10:48.906 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout [global] 2023-06-23T01:10:48.906 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout fsid = 9efbfa8e-1162-11ee-9b2b-001a4aab830c 2023-06-23T01:10:48.907 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.120:3300,v1:172.21.15.120:6789] 2023-06-23T01:10:48.907 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-06-23T01:10:48.907 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-06-23T01:10:48.907 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-06-23T01:10:48.907 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-06-23T01:10:48.907 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 2023-06-23T01:10:48.907 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout [mgr] 2023-06-23T01:10:48.908 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-06-23T01:10:48.908 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 2023-06-23T01:10:48.908 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout [osd] 2023-06-23T01:10:48.908 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-06-23T01:10:48.908 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-06-23T01:10:48.908 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-06-23T01:10:48.908 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-06-23T01:10:48.908 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-06-23T01:10:48.912 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.460+0000 7f4fef59e700 1 Processor -- start 2023-06-23T01:10:48.912 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.460+0000 7f4fef59e700 1 -- start start 2023-06-23T01:10:48.912 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.460+0000 7f4fef59e700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ff0077c20 0x7f4ff0076130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:10:48.912 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.460+0000 7f4fef59e700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff0078030 con 0x7f4ff0077c20 2023-06-23T01:10:48.913 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.460+0000 7f4fee59c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ff0077c20 0x7f4ff0076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:10:48.913 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.460+0000 7f4fee59c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ff0077c20 0x7f4ff0076130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58956/0 (socket says 172.21.15.120:58956) 2023-06-23T01:10:48.913 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.460+0000 7f4fee59c700 1 -- 172.21.15.120:0/30369244 learned_addr learned my addr 172.21.15.120:0/30369244 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:10:48.913 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.461+0000 7f4fee59c700 1 -- 172.21.15.120:0/30369244 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff0076670 con 0x7f4ff0077c20 2023-06-23T01:10:48.913 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.461+0000 7f4fee59c700 1 --2- 172.21.15.120:0/30369244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ff0077c20 0x7f4ff0076130 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0009e50 tx=0x7f4fe0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd36fa52f40037fb server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:10:48.913 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.461+0000 7f4fed59a700 1 -- 172.21.15.120:0/30369244 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4fe000e070 con 0x7f4ff0077c20 2023-06-23T01:10:48.913 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.461+0000 7f4fed59a700 1 -- 172.21.15.120:0/30369244 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe000ba20 con 0x7f4ff0077c20 2023-06-23T01:10:48.913 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.461+0000 7f4fed59a700 1 -- 172.21.15.120:0/30369244 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4fe00092e0 con 0x7f4ff0077c20 2023-06-23T01:10:48.914 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.462+0000 7f4fef59e700 1 -- 172.21.15.120:0/30369244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ff0077c20 msgr2=0x7f4ff0076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:10:48.914 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.462+0000 7f4fef59e700 1 --2- 172.21.15.120:0/30369244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ff0077c20 0x7f4ff0076130 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0009e50 tx=0x7f4fe0009840 comp rx=0 tx=0).stop 2023-06-23T01:10:48.914 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.462+0000 7f4fef59e700 1 -- 172.21.15.120:0/30369244 shutdown_connections 2023-06-23T01:10:48.914 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.462+0000 7f4fef59e700 1 --2- 172.21.15.120:0/30369244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ff0077c20 0x7f4ff0076130 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:10:48.914 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.462+0000 7f4fef59e700 1 -- 172.21.15.120:0/30369244 >> 172.21.15.120:0/30369244 conn(0x7f4ff00fd810 msgr2=0x7f4ff00ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:10:48.914 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.462+0000 7f4fef59e700 1 -- 172.21.15.120:0/30369244 shutdown_connections 2023-06-23T01:10:48.914 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.462+0000 7f4fef59e700 1 -- 172.21.15.120:0/30369244 wait complete. 2023-06-23T01:10:48.915 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.463+0000 7f4fef59e700 1 Processor -- start 2023-06-23T01:10:48.915 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.463+0000 7f4fef59e700 1 -- start start 2023-06-23T01:10:48.915 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.463+0000 7f4fef59e700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ff0077c20 0x7f4ff01127b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:10:48.915 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.463+0000 7f4fef59e700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff0112cf0 con 0x7f4ff0077c20 2023-06-23T01:10:48.915 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.463+0000 7f4fee59c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ff0077c20 0x7f4ff01127b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:10:48.915 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.463+0000 7f4fee59c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ff0077c20 0x7f4ff01127b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58958/0 (socket says 172.21.15.120:58958) 2023-06-23T01:10:48.915 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.463+0000 7f4fee59c700 1 -- 172.21.15.120:0/1233880534 learned_addr learned my addr 172.21.15.120:0/1233880534 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:10:48.915 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.464+0000 7f4fee59c700 1 -- 172.21.15.120:0/1233880534 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe0009ae0 con 0x7f4ff0077c20 2023-06-23T01:10:48.916 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.464+0000 7f4fee59c700 1 --2- 172.21.15.120:0/1233880534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ff0077c20 0x7f4ff01127b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0003000 tx=0x7f4fe0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:10:48.916 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.464+0000 7f4fdeffd700 1 -- 172.21.15.120:0/1233880534 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4fe00092e0 con 0x7f4ff0077c20 2023-06-23T01:10:48.916 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.464+0000 7f4fdeffd700 1 -- 172.21.15.120:0/1233880534 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe0012b90 con 0x7f4ff0077c20 2023-06-23T01:10:48.916 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.464+0000 7f4fdeffd700 1 -- 172.21.15.120:0/1233880534 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4fe0012d60 con 0x7f4ff0077c20 2023-06-23T01:10:48.916 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.464+0000 7f4fef59e700 1 -- 172.21.15.120:0/1233880534 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ff0112ef0 con 0x7f4ff0077c20 2023-06-23T01:10:48.916 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.464+0000 7f4fef59e700 1 -- 172.21.15.120:0/1233880534 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ff0113b90 con 0x7f4ff0077c20 2023-06-23T01:10:48.916 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.465+0000 7f4fdeffd700 1 -- 172.21.15.120:0/1233880534 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f4fe0019030 con 0x7f4ff0077c20 2023-06-23T01:10:48.917 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.465+0000 7f4fdeffd700 1 -- 172.21.15.120:0/1233880534 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f4fe001ea40 con 0x7f4ff0077c20 2023-06-23T01:10:48.917 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.465+0000 7f4fef59e700 1 -- 172.21.15.120:0/1233880534 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ff005f060 con 0x7f4ff0077c20 2023-06-23T01:10:48.917 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.468+0000 7f4fdeffd700 1 -- 172.21.15.120:0/1233880534 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f4fe0017070 con 0x7f4ff0077c20 2023-06-23T01:10:48.917 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.537+0000 7f4fef59e700 1 -- 172.21.15.120:0/1233880534 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f4ff0076b70 con 0x7f4ff0077c20 2023-06-23T01:10:48.917 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.542+0000 7f4fdeffd700 1 -- 172.21.15.120:0/1233880534 <== mon.0 v2:172.21.15.120:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4fe001e400 con 0x7f4ff0077c20 2023-06-23T01:10:48.917 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.542+0000 7f4fdeffd700 1 -- 172.21.15.120:0/1233880534 <== mon.0 v2:172.21.15.120:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+431 (secure 0 0 0) 0x7f4ff0076b70 con 0x7f4ff0077c20 2023-06-23T01:10:48.917 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.543+0000 7f4fef59e700 1 -- 172.21.15.120:0/1233880534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ff0077c20 msgr2=0x7f4ff01127b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:10:48.917 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.543+0000 7f4fef59e700 1 --2- 172.21.15.120:0/1233880534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ff0077c20 0x7f4ff01127b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0003000 tx=0x7f4fe0017000 comp rx=0 tx=0).stop 2023-06-23T01:10:48.918 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.543+0000 7f4fef59e700 1 -- 172.21.15.120:0/1233880534 shutdown_connections 2023-06-23T01:10:48.918 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.543+0000 7f4fef59e700 1 --2- 172.21.15.120:0/1233880534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ff0077c20 0x7f4ff01127b0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:10:48.918 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.543+0000 7f4fef59e700 1 -- 172.21.15.120:0/1233880534 >> 172.21.15.120:0/1233880534 conn(0x7f4ff00fd810 msgr2=0x7f4ff0106d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:10:48.918 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.543+0000 7f4fef59e700 1 -- 172.21.15.120:0/1233880534 shutdown_connections 2023-06-23T01:10:48.918 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:48.543+0000 7f4fef59e700 1 -- 172.21.15.120:0/1233880534 wait complete. 2023-06-23T01:10:48.918 INFO:teuthology.orchestra.run.smithi120.stdout:Generating new minimal ceph.conf... 2023-06-23T01:10:49.149 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:48 smithi120 ceph-mon[78902]: from='client.? 172.21.15.120:0/1233880534' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-06-23T01:10:49.150 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:48 smithi120 ceph-mon[78902]: from='client.? 172.21.15.120:0/1233880534' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-06-23T01:10:49.879 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.391+0000 7f59a9672700 1 Processor -- start 2023-06-23T01:10:49.882 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.393+0000 7f59a9672700 1 -- start start 2023-06-23T01:10:49.883 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.393+0000 7f59a9672700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59a4104470 0x7f59a4104880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:10:49.883 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.393+0000 7f59a9672700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a4104dc0 con 0x7f59a4104470 2023-06-23T01:10:49.883 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.393+0000 7f59a2ffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59a4104470 0x7f59a4104880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:10:49.883 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.393+0000 7f59a2ffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59a4104470 0x7f59a4104880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58972/0 (socket says 172.21.15.120:58972) 2023-06-23T01:10:49.883 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.393+0000 7f59a2ffd700 1 -- 172.21.15.120:0/2636097794 learned_addr learned my addr 172.21.15.120:0/2636097794 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:10:49.883 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.393+0000 7f59a2ffd700 1 -- 172.21.15.120:0/2636097794 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a4104f00 con 0x7f59a4104470 2023-06-23T01:10:49.883 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.393+0000 7f59a2ffd700 1 --2- 172.21.15.120:0/2636097794 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59a4104470 0x7f59a4104880 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f59940098e0 tx=0x7f5994009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc30916f40815d6 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:10:49.884 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.393+0000 7f59a27fc700 1 -- 172.21.15.120:0/2636097794 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5994009e80 con 0x7f59a4104470 2023-06-23T01:10:49.884 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.393+0000 7f59a27fc700 1 -- 172.21.15.120:0/2636097794 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f599400bd10 con 0x7f59a4104470 2023-06-23T01:10:49.884 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.393+0000 7f59a27fc700 1 -- 172.21.15.120:0/2636097794 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5994011ce0 con 0x7f59a4104470 2023-06-23T01:10:49.884 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.394+0000 7f59a9672700 1 -- 172.21.15.120:0/2636097794 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59a4104470 msgr2=0x7f59a4104880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:10:49.884 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.394+0000 7f59a9672700 1 --2- 172.21.15.120:0/2636097794 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59a4104470 0x7f59a4104880 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f59940098e0 tx=0x7f5994009250 comp rx=0 tx=0).stop 2023-06-23T01:10:49.884 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.394+0000 7f59a9672700 1 -- 172.21.15.120:0/2636097794 shutdown_connections 2023-06-23T01:10:49.884 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.394+0000 7f59a9672700 1 --2- 172.21.15.120:0/2636097794 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59a4104470 0x7f59a4104880 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:10:49.885 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.394+0000 7f59a9672700 1 -- 172.21.15.120:0/2636097794 >> 172.21.15.120:0/2636097794 conn(0x7f59a4077a20 msgr2=0x7f59a41023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:10:49.885 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.394+0000 7f59a9672700 1 -- 172.21.15.120:0/2636097794 shutdown_connections 2023-06-23T01:10:49.885 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.394+0000 7f59a9672700 1 -- 172.21.15.120:0/2636097794 wait complete. 2023-06-23T01:10:49.885 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.395+0000 7f59a9672700 1 Processor -- start 2023-06-23T01:10:49.885 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.395+0000 7f59a9672700 1 -- start start 2023-06-23T01:10:49.885 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.395+0000 7f59a9672700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59a4104470 0x7f59a41130e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:10:49.885 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.395+0000 7f59a9672700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a4113620 con 0x7f59a4104470 2023-06-23T01:10:49.885 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.395+0000 7f59a2ffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59a4104470 0x7f59a41130e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:10:49.886 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.395+0000 7f59a2ffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59a4104470 0x7f59a41130e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58982/0 (socket says 172.21.15.120:58982) 2023-06-23T01:10:49.886 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.395+0000 7f59a2ffd700 1 -- 172.21.15.120:0/1084678389 learned_addr learned my addr 172.21.15.120:0/1084678389 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:10:49.886 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.396+0000 7f59a2ffd700 1 -- 172.21.15.120:0/1084678389 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59940094f0 con 0x7f59a4104470 2023-06-23T01:10:49.886 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.396+0000 7f59a2ffd700 1 --2- 172.21.15.120:0/1084678389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59a4104470 0x7f59a41130e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5994006b00 tx=0x7f5994009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:10:49.886 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.396+0000 7f599b7fe700 1 -- 172.21.15.120:0/1084678389 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f599401a640 con 0x7f59a4104470 2023-06-23T01:10:49.886 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.396+0000 7f599b7fe700 1 -- 172.21.15.120:0/1084678389 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f599401ac50 con 0x7f59a4104470 2023-06-23T01:10:49.886 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.396+0000 7f59a9672700 1 -- 172.21.15.120:0/1084678389 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59a4113820 con 0x7f59a4104470 2023-06-23T01:10:49.886 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.396+0000 7f599b7fe700 1 -- 172.21.15.120:0/1084678389 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5994022ab0 con 0x7f59a4104470 2023-06-23T01:10:49.887 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.396+0000 7f59a9672700 1 -- 172.21.15.120:0/1084678389 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59a41144c0 con 0x7f59a4104470 2023-06-23T01:10:49.887 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.396+0000 7f599b7fe700 1 -- 172.21.15.120:0/1084678389 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f599402c440 con 0x7f59a4104470 2023-06-23T01:10:49.887 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.396+0000 7f599b7fe700 1 -- 172.21.15.120:0/1084678389 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f599402b730 con 0x7f59a4104470 2023-06-23T01:10:49.887 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.397+0000 7f59a9672700 1 -- 172.21.15.120:0/1084678389 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59a405f060 con 0x7f59a4104470 2023-06-23T01:10:49.887 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.400+0000 7f599b7fe700 1 -- 172.21.15.120:0/1084678389 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f599401adc0 con 0x7f59a4104470 2023-06-23T01:10:49.887 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.469+0000 7f59a9672700 1 -- 172.21.15.120:0/1084678389 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f59a41062c0 con 0x7f59a4104470 2023-06-23T01:10:49.887 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.470+0000 7f599b7fe700 1 -- 172.21.15.120:0/1084678389 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7f5994022400 con 0x7f59a4104470 2023-06-23T01:10:49.888 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.471+0000 7f59a9672700 1 -- 172.21.15.120:0/1084678389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59a4104470 msgr2=0x7f59a41130e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:10:49.888 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.471+0000 7f59a9672700 1 --2- 172.21.15.120:0/1084678389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59a4104470 0x7f59a41130e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5994006b00 tx=0x7f5994009250 comp rx=0 tx=0).stop 2023-06-23T01:10:49.888 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.471+0000 7f59a9672700 1 -- 172.21.15.120:0/1084678389 shutdown_connections 2023-06-23T01:10:49.888 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.471+0000 7f59a9672700 1 --2- 172.21.15.120:0/1084678389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59a4104470 0x7f59a41130e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:10:49.888 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.471+0000 7f59a9672700 1 -- 172.21.15.120:0/1084678389 >> 172.21.15.120:0/1084678389 conn(0x7f59a4077a20 msgr2=0x7f59a4101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:10:49.888 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.471+0000 7f59a9672700 1 -- 172.21.15.120:0/1084678389 shutdown_connections 2023-06-23T01:10:49.888 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:49.471+0000 7f59a9672700 1 -- 172.21.15.120:0/1084678389 wait complete. 2023-06-23T01:10:49.888 INFO:teuthology.orchestra.run.smithi120.stdout:Restarting the monitor... 2023-06-23T01:10:50.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:49 smithi120 ceph-mon[78902]: from='client.? 172.21.15.120:0/1084678389' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:10:50.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:49 smithi120 systemd[1]: Stopping Ceph mon.a for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:10:50.630 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:50 smithi120 conmon[78878]: 2023-06-23T01:10:50.346+0000 7fc0e5bbe700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T01:10:50.630 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:50 smithi120 conmon[78878]: 2023-06-23T01:10:50.346+0000 7fc0e5bbe700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-06-23T01:10:50.988 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:50 smithi120 bash[79432]: ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c-mon-a 2023-06-23T01:10:51.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 systemd[1]: ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mon.a.service: Succeeded. 2023-06-23T01:10:51.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 systemd[1]: Stopped Ceph mon.a for 9efbfa8e-1162-11ee-9b2b-001a4aab830c. 2023-06-23T01:10:51.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 systemd[1]: Starting Ceph mon.a for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:10:51.906 INFO:teuthology.orchestra.run.smithi120.stdout:Setting mon public_network to 172.21.0.0/20 2023-06-23T01:10:52.155 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: set uid:gid to 167:167 (ceph:ceph) 2023-06-23T01:10:52.155 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: ceph version 17.2.6-719-g42ddc46e (42ddc46e6e6582d8e29a5a982a81886fe028be2c) quincy (stable), process ceph-mon, pid 7 2023-06-23T01:10:52.155 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: pidfile_write: ignore empty --pid-file 2023-06-23T01:10:52.155 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: load: jerasure load: lrc 2023-06-23T01:10:52.156 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: RocksDB version: 6.15.5 2023-06-23T01:10:52.156 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2023-06-23T01:10:52.156 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Compile date Jun 21 2023 2023-06-23T01:10:52.156 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: DB SUMMARY 2023-06-23T01:10:52.156 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: DB Session ID: 1GXL5Y2U4PP6IV4SU94X 2023-06-23T01:10:52.156 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: CURRENT file: CURRENT 2023-06-23T01:10:52.156 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: IDENTITY file: IDENTITY 2023-06-23T01:10:52.157 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2023-06-23T01:10:52.157 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-06-23T01:10:52.157 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000010.log size: 97730 ; 2023-06-23T01:10:52.157 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.error_if_exists: 0 2023-06-23T01:10:52.157 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.create_if_missing: 0 2023-06-23T01:10:52.157 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.paranoid_checks: 1 2023-06-23T01:10:52.157 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-06-23T01:10:52.158 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.env: 0x56403f4f3a80 2023-06-23T01:10:52.158 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.fs: Posix File System 2023-06-23T01:10:52.158 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.info_log: 0x56403fd83da0 2023-06-23T01:10:52.158 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.max_file_opening_threads: 16 2023-06-23T01:10:52.158 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.statistics: (nil) 2023-06-23T01:10:52.158 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.use_fsync: 0 2023-06-23T01:10:52.158 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.max_log_file_size: 0 2023-06-23T01:10:52.158 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-06-23T01:10:52.159 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.log_file_time_to_roll: 0 2023-06-23T01:10:52.159 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.keep_log_file_num: 1000 2023-06-23T01:10:52.159 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.recycle_log_file_num: 0 2023-06-23T01:10:52.159 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.allow_fallocate: 1 2023-06-23T01:10:52.159 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.allow_mmap_reads: 0 2023-06-23T01:10:52.159 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.allow_mmap_writes: 0 2023-06-23T01:10:52.159 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.use_direct_reads: 0 2023-06-23T01:10:52.159 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-06-23T01:10:52.160 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.create_missing_column_families: 0 2023-06-23T01:10:52.160 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.db_log_dir: 2023-06-23T01:10:52.160 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2023-06-23T01:10:52.160 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.table_cache_numshardbits: 6 2023-06-23T01:10:52.160 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.WAL_ttl_seconds: 0 2023-06-23T01:10:52.160 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.WAL_size_limit_MB: 0 2023-06-23T01:10:52.160 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-06-23T01:10:52.161 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-06-23T01:10:52.161 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.is_fd_close_on_exec: 1 2023-06-23T01:10:52.161 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.advise_random_on_open: 1 2023-06-23T01:10:52.161 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.db_write_buffer_size: 0 2023-06-23T01:10:52.161 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.write_buffer_manager: 0x56403fe488d0 2023-06-23T01:10:52.161 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-06-23T01:10:52.161 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2023-06-23T01:10:52.161 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-06-23T01:10:52.162 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.use_adaptive_mutex: 0 2023-06-23T01:10:52.162 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.rate_limiter: (nil) 2023-06-23T01:10:52.162 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-06-23T01:10:52.162 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.wal_recovery_mode: 2 2023-06-23T01:10:52.162 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.enable_thread_tracking: 0 2023-06-23T01:10:52.162 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.enable_pipelined_write: 0 2023-06-23T01:10:52.162 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.unordered_write: 0 2023-06-23T01:10:52.162 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-06-23T01:10:52.163 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-06-23T01:10:52.163 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-06-23T01:10:52.163 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-06-23T01:10:52.163 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.row_cache: None 2023-06-23T01:10:52.163 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.wal_filter: None 2023-06-23T01:10:52.163 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-06-23T01:10:52.163 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.allow_ingest_behind: 0 2023-06-23T01:10:52.163 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.preserve_deletes: 0 2023-06-23T01:10:52.164 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.two_write_queues: 0 2023-06-23T01:10:52.164 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.manual_wal_flush: 0 2023-06-23T01:10:52.164 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.atomic_flush: 0 2023-06-23T01:10:52.164 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-06-23T01:10:52.164 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.persist_stats_to_disk: 0 2023-06-23T01:10:52.164 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.write_dbid_to_manifest: 0 2023-06-23T01:10:52.164 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.log_readahead_size: 0 2023-06-23T01:10:52.164 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-06-23T01:10:52.165 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.best_efforts_recovery: 0 2023-06-23T01:10:52.165 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-06-23T01:10:52.165 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-06-23T01:10:52.165 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.allow_data_in_errors: 0 2023-06-23T01:10:52.165 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.db_host_id: __hostname__ 2023-06-23T01:10:52.165 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.max_background_jobs: 2 2023-06-23T01:10:52.165 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.max_background_compactions: -1 2023-06-23T01:10:52.165 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.max_subcompactions: 1 2023-06-23T01:10:52.166 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-06-23T01:10:52.166 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-06-23T01:10:52.166 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.delayed_write_rate : 16777216 2023-06-23T01:10:52.166 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.max_total_wal_size: 0 2023-06-23T01:10:52.166 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-06-23T01:10:52.166 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.stats_dump_period_sec: 600 2023-06-23T01:10:52.166 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.stats_persist_period_sec: 600 2023-06-23T01:10:52.166 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-06-23T01:10:52.167 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.max_open_files: -1 2023-06-23T01:10:52.167 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.bytes_per_sync: 0 2023-06-23T01:10:52.167 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.wal_bytes_per_sync: 0 2023-06-23T01:10:52.167 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.strict_bytes_per_sync: 0 2023-06-23T01:10:52.167 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.compaction_readahead_size: 0 2023-06-23T01:10:52.167 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.max_background_flushes: -1 2023-06-23T01:10:52.167 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Compression algorithms supported: 2023-06-23T01:10:52.168 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-06-23T01:10:52.168 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: kZSTD supported: 0 2023-06-23T01:10:52.168 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: kXpressCompression supported: 0 2023-06-23T01:10:52.168 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: kLZ4HCCompression supported: 1 2023-06-23T01:10:52.168 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: kZlibCompression supported: 1 2023-06-23T01:10:52.168 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: kSnappyCompression supported: 1 2023-06-23T01:10:52.168 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: kLZ4Compression supported: 1 2023-06-23T01:10:52.168 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: kBZip2Compression supported: 0 2023-06-23T01:10:52.169 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Fast CRC32 supported: Supported on x86 2023-06-23T01:10:52.169 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 2023-06-23T01:10:52.169 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2023-06-23T01:10:52.169 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-06-23T01:10:52.169 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.merge_operator: 2023-06-23T01:10:52.169 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.compaction_filter: None 2023-06-23T01:10:52.169 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.compaction_filter_factory: None 2023-06-23T01:10:52.169 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.sst_partitioner_factory: None 2023-06-23T01:10:52.170 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.memtable_factory: SkipListFactory 2023-06-23T01:10:52.170 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.table_factory: BlockBasedTable 2023-06-23T01:10:52.170 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56403fd52d30) 2023-06-23T01:10:52.170 INFO:journalctl@ceph.mon.a.smithi120.stdout: cache_index_and_filter_blocks: 1 2023-06-23T01:10:52.170 INFO:journalctl@ceph.mon.a.smithi120.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-06-23T01:10:52.170 INFO:journalctl@ceph.mon.a.smithi120.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-06-23T01:10:52.170 INFO:journalctl@ceph.mon.a.smithi120.stdout: pin_top_level_index_and_filter: 1 2023-06-23T01:10:52.170 INFO:journalctl@ceph.mon.a.smithi120.stdout: index_type: 0 2023-06-23T01:10:52.171 INFO:journalctl@ceph.mon.a.smithi120.stdout: data_block_index_type: 0 2023-06-23T01:10:52.172 INFO:journalctl@ceph.mon.a.smithi120.stdout: index_shortening: 1 2023-06-23T01:10:52.172 INFO:journalctl@ceph.mon.a.smithi120.stdout: data_block_hash_table_util_ratio: 0.750000 2023-06-23T01:10:52.172 INFO:journalctl@ceph.mon.a.smithi120.stdout: hash_index_allow_collision: 1 2023-06-23T01:10:52.172 INFO:journalctl@ceph.mon.a.smithi120.stdout: checksum: 1 2023-06-23T01:10:52.172 INFO:journalctl@ceph.mon.a.smithi120.stdout: no_block_cache: 0 2023-06-23T01:10:52.172 INFO:journalctl@ceph.mon.a.smithi120.stdout: block_cache: 0x56403fe0b1f0 2023-06-23T01:10:52.172 INFO:journalctl@ceph.mon.a.smithi120.stdout: block_cache_name: BinnedLRUCache 2023-06-23T01:10:52.173 INFO:journalctl@ceph.mon.a.smithi120.stdout: block_cache_options: 2023-06-23T01:10:52.173 INFO:journalctl@ceph.mon.a.smithi120.stdout: capacity : 536870912 2023-06-23T01:10:52.173 INFO:journalctl@ceph.mon.a.smithi120.stdout: num_shard_bits : 4 2023-06-23T01:10:52.173 INFO:journalctl@ceph.mon.a.smithi120.stdout: strict_capacity_limit : 0 2023-06-23T01:10:52.173 INFO:journalctl@ceph.mon.a.smithi120.stdout: high_pri_pool_ratio: 0.000 2023-06-23T01:10:52.173 INFO:journalctl@ceph.mon.a.smithi120.stdout: block_cache_compressed: (nil) 2023-06-23T01:10:52.173 INFO:journalctl@ceph.mon.a.smithi120.stdout: persistent_cache: (nil) 2023-06-23T01:10:52.174 INFO:journalctl@ceph.mon.a.smithi120.stdout: block_size: 4096 2023-06-23T01:10:52.174 INFO:journalctl@ceph.mon.a.smithi120.stdout: block_size_deviation: 10 2023-06-23T01:10:52.174 INFO:journalctl@ceph.mon.a.smithi120.stdout: block_restart_interval: 16 2023-06-23T01:10:52.174 INFO:journalctl@ceph.mon.a.smithi120.stdout: index_block_restart_interval: 1 2023-06-23T01:10:52.174 INFO:journalctl@ceph.mon.a.smithi120.stdout: metadata_block_size: 4096 2023-06-23T01:10:52.174 INFO:journalctl@ceph.mon.a.smithi120.stdout: partition_filters: 0 2023-06-23T01:10:52.174 INFO:journalctl@ceph.mon.a.smithi120.stdout: use_delta_encoding: 1 2023-06-23T01:10:52.174 INFO:journalctl@ceph.mon.a.smithi120.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2023-06-23T01:10:52.175 INFO:journalctl@ceph.mon.a.smithi120.stdout: whole_key_filtering: 1 2023-06-23T01:10:52.175 INFO:journalctl@ceph.mon.a.smithi120.stdout: verify_compression: 0 2023-06-23T01:10:52.175 INFO:journalctl@ceph.mon.a.smithi120.stdout: read_amp_bytes_per_bit: 0 2023-06-23T01:10:52.175 INFO:journalctl@ceph.mon.a.smithi120.stdout: format_version: 4 2023-06-23T01:10:52.175 INFO:journalctl@ceph.mon.a.smithi120.stdout: enable_index_compression: 1 2023-06-23T01:10:52.175 INFO:journalctl@ceph.mon.a.smithi120.stdout: block_align: 0 2023-06-23T01:10:52.175 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.write_buffer_size: 33554432 2023-06-23T01:10:52.175 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.max_write_buffer_number: 2 2023-06-23T01:10:52.176 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.compression: NoCompression 2023-06-23T01:10:52.176 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.bottommost_compression: Disabled 2023-06-23T01:10:52.176 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.prefix_extractor: nullptr 2023-06-23T01:10:52.176 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-06-23T01:10:52.176 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.num_levels: 7 2023-06-23T01:10:52.176 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-06-23T01:10:52.176 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-06-23T01:10:52.176 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-06-23T01:10:52.177 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-06-23T01:10:52.177 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-06-23T01:10:52.177 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-06-23T01:10:52.177 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-06-23T01:10:52.177 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-06-23T01:10:52.177 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-06-23T01:10:52.177 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-06-23T01:10:52.178 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.compression_opts.window_bits: -14 2023-06-23T01:10:52.178 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.compression_opts.level: 32767 2023-06-23T01:10:52.178 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.compression_opts.strategy: 0 2023-06-23T01:10:52.178 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-06-23T01:10:52.178 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-06-23T01:10:52.178 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-06-23T01:10:52.178 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.compression_opts.enabled: false 2023-06-23T01:10:52.178 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-06-23T01:10:52.179 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-06-23T01:10:52.179 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-06-23T01:10:52.179 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.target_file_size_base: 67108864 2023-06-23T01:10:52.179 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.target_file_size_multiplier: 1 2023-06-23T01:10:52.179 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-06-23T01:10:52.179 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-06-23T01:10:52.179 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-06-23T01:10:52.179 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-06-23T01:10:52.180 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-06-23T01:10:52.180 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-06-23T01:10:52.180 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-06-23T01:10:52.180 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-06-23T01:10:52.180 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-06-23T01:10:52.180 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-06-23T01:10:52.180 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-06-23T01:10:52.180 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-06-23T01:10:52.181 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.arena_block_size: 4194304 2023-06-23T01:10:52.181 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-06-23T01:10:52.181 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-06-23T01:10:52.181 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2023-06-23T01:10:52.181 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.disable_auto_compactions: 0 2023-06-23T01:10:52.181 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-06-23T01:10:52.181 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-06-23T01:10:52.181 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-06-23T01:10:52.182 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-06-23T01:10:52.182 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-06-23T01:10:52.182 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-06-23T01:10:52.182 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-06-23T01:10:52.182 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-06-23T01:10:52.182 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-06-23T01:10:52.182 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-06-23T01:10:52.182 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.table_properties_collectors: 2023-06-23T01:10:52.183 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.inplace_update_support: 0 2023-06-23T01:10:52.183 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.inplace_update_num_locks: 10000 2023-06-23T01:10:52.183 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-06-23T01:10:52.183 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-06-23T01:10:52.183 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.memtable_huge_page_size: 0 2023-06-23T01:10:52.183 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.bloom_locality: 0 2023-06-23T01:10:52.183 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.max_successive_merges: 0 2023-06-23T01:10:52.184 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.optimize_filters_for_hits: 0 2023-06-23T01:10:52.184 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.paranoid_file_checks: 0 2023-06-23T01:10:52.184 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.force_consistency_checks: 1 2023-06-23T01:10:52.184 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.report_bg_io_stats: 0 2023-06-23T01:10:52.184 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.ttl: 2592000 2023-06-23T01:10:52.184 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.periodic_compaction_seconds: 0 2023-06-23T01:10:52.184 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.enable_blob_files: false 2023-06-23T01:10:52.184 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.min_blob_size: 0 2023-06-23T01:10:52.185 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.blob_file_size: 268435456 2023-06-23T01:10:52.185 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.blob_compression_type: NoCompression 2023-06-23T01:10:52.185 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.enable_blob_garbage_collection: false 2023-06-23T01:10:52.185 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-06-23T01:10:52.185 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-06-23T01:10:52.185 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2023-06-23T01:10:52.185 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2023-06-23T01:10:52.185 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: EVENT_LOG_v1 {"time_micros": 1687482651866872, "job": 1, "event": "recovery_started", "wal_files": [10]} 2023-06-23T01:10:52.186 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2023-06-23T01:10:52.186 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2023-06-23T01:10:52.186 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: EVENT_LOG_v1 {"time_micros": 1687482651869413, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 92992, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 91070, "index_size": 282, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13491, "raw_average_key_size": 49, "raw_value_size": 83687, "raw_average_value_size": 308, "num_data_blocks": 11, "num_entries": 271, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1687482651, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "41aa9914-a775-4aad-ba97-7a06436a92a9", "db_session_id": "1GXL5Y2U4PP6IV4SU94X"}} 2023-06-23T01:10:52.186 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2023-06-23T01:10:52.186 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: EVENT_LOG_v1 {"time_micros": 1687482651870337, "job": 1, "event": "recovery_finished"} 2023-06-23T01:10:52.186 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-06-23T01:10:52.186 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x56403fd9f180 2023-06-23T01:10:52.187 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: DB pointer 0x56403fe14000 2023-06-23T01:10:52.187 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2023-06-23T01:10:52.187 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: rocksdb: [db/db_impl/db_impl.cc:903] 2023-06-23T01:10:52.187 INFO:journalctl@ceph.mon.a.smithi120.stdout: ** DB Stats ** 2023-06-23T01:10:52.187 INFO:journalctl@ceph.mon.a.smithi120.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-06-23T01:10:52.187 INFO:journalctl@ceph.mon.a.smithi120.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-06-23T01:10:52.187 INFO:journalctl@ceph.mon.a.smithi120.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-06-23T01:10:52.187 INFO:journalctl@ceph.mon.a.smithi120.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-06-23T01:10:52.188 INFO:journalctl@ceph.mon.a.smithi120.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-06-23T01:10:52.188 INFO:journalctl@ceph.mon.a.smithi120.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2023-06-23T01:10:52.188 INFO:journalctl@ceph.mon.a.smithi120.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-06-23T01:10:52.188 INFO:journalctl@ceph.mon.a.smithi120.stdout: 2023-06-23T01:10:52.188 INFO:journalctl@ceph.mon.a.smithi120.stdout: ** Compaction Stats [default] ** 2023-06-23T01:10:52.188 INFO:journalctl@ceph.mon.a.smithi120.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-06-23T01:10:52.188 INFO:journalctl@ceph.mon.a.smithi120.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-06-23T01:10:52.188 INFO:journalctl@ceph.mon.a.smithi120.stdout: L0 2/0 92.57 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 50.6 0.00 0.00 1 0.002 0 0 2023-06-23T01:10:52.189 INFO:journalctl@ceph.mon.a.smithi120.stdout: Sum 2/0 92.57 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 50.6 0.00 0.00 1 0.002 0 0 2023-06-23T01:10:52.189 INFO:journalctl@ceph.mon.a.smithi120.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 50.6 0.00 0.00 1 0.002 0 0 2023-06-23T01:10:52.189 INFO:journalctl@ceph.mon.a.smithi120.stdout: 2023-06-23T01:10:52.189 INFO:journalctl@ceph.mon.a.smithi120.stdout: ** Compaction Stats [default] ** 2023-06-23T01:10:52.189 INFO:journalctl@ceph.mon.a.smithi120.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-06-23T01:10:52.189 INFO:journalctl@ceph.mon.a.smithi120.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-06-23T01:10:52.189 INFO:journalctl@ceph.mon.a.smithi120.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 50.6 0.00 0.00 1 0.002 0 0 2023-06-23T01:10:52.189 INFO:journalctl@ceph.mon.a.smithi120.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-06-23T01:10:52.190 INFO:journalctl@ceph.mon.a.smithi120.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-06-23T01:10:52.190 INFO:journalctl@ceph.mon.a.smithi120.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-06-23T01:10:52.190 INFO:journalctl@ceph.mon.a.smithi120.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-06-23T01:10:52.190 INFO:journalctl@ceph.mon.a.smithi120.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-06-23T01:10:52.190 INFO:journalctl@ceph.mon.a.smithi120.stdout: AddFile(Keys): cumulative 0, interval 0 2023-06-23T01:10:52.190 INFO:journalctl@ceph.mon.a.smithi120.stdout: Cumulative compaction: 0.00 GB write, 9.32 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-06-23T01:10:52.190 INFO:journalctl@ceph.mon.a.smithi120.stdout: Interval compaction: 0.00 GB write, 9.32 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-06-23T01:10:52.191 INFO:journalctl@ceph.mon.a.smithi120.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-06-23T01:10:52.191 INFO:journalctl@ceph.mon.a.smithi120.stdout: 2023-06-23T01:10:52.191 INFO:journalctl@ceph.mon.a.smithi120.stdout: ** File Read Latency Histogram By Level [default] ** 2023-06-23T01:10:52.191 INFO:journalctl@ceph.mon.a.smithi120.stdout: 2023-06-23T01:10:52.191 INFO:journalctl@ceph.mon.a.smithi120.stdout: ** Compaction Stats [default] ** 2023-06-23T01:10:52.191 INFO:journalctl@ceph.mon.a.smithi120.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-06-23T01:10:52.191 INFO:journalctl@ceph.mon.a.smithi120.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-06-23T01:10:52.191 INFO:journalctl@ceph.mon.a.smithi120.stdout: L0 2/0 92.57 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 50.6 0.00 0.00 1 0.002 0 0 2023-06-23T01:10:52.192 INFO:journalctl@ceph.mon.a.smithi120.stdout: Sum 2/0 92.57 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 50.6 0.00 0.00 1 0.002 0 0 2023-06-23T01:10:52.192 INFO:journalctl@ceph.mon.a.smithi120.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2023-06-23T01:10:52.192 INFO:journalctl@ceph.mon.a.smithi120.stdout: 2023-06-23T01:10:52.192 INFO:journalctl@ceph.mon.a.smithi120.stdout: ** Compaction Stats [default] ** 2023-06-23T01:10:52.192 INFO:journalctl@ceph.mon.a.smithi120.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-06-23T01:10:52.192 INFO:journalctl@ceph.mon.a.smithi120.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-06-23T01:10:52.192 INFO:journalctl@ceph.mon.a.smithi120.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 50.6 0.00 0.00 1 0.002 0 0 2023-06-23T01:10:52.192 INFO:journalctl@ceph.mon.a.smithi120.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-06-23T01:10:52.193 INFO:journalctl@ceph.mon.a.smithi120.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-06-23T01:10:52.193 INFO:journalctl@ceph.mon.a.smithi120.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-06-23T01:10:52.193 INFO:journalctl@ceph.mon.a.smithi120.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-06-23T01:10:52.193 INFO:journalctl@ceph.mon.a.smithi120.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-06-23T01:10:52.193 INFO:journalctl@ceph.mon.a.smithi120.stdout: AddFile(Keys): cumulative 0, interval 0 2023-06-23T01:10:52.193 INFO:journalctl@ceph.mon.a.smithi120.stdout: Cumulative compaction: 0.00 GB write, 9.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-06-23T01:10:52.193 INFO:journalctl@ceph.mon.a.smithi120.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-06-23T01:10:52.194 INFO:journalctl@ceph.mon.a.smithi120.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-06-23T01:10:52.194 INFO:journalctl@ceph.mon.a.smithi120.stdout: 2023-06-23T01:10:52.194 INFO:journalctl@ceph.mon.a.smithi120.stdout: ** File Read Latency Histogram By Level [default] ** 2023-06-23T01:10:52.194 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: starting mon.a rank 0 at public addrs [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] at bind addrs [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c 2023-06-23T01:10:52.194 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: mon.a@-1(???) e1 preinit fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c 2023-06-23T01:10:52.194 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: mon.a@-1(???).mds e1 new map 2023-06-23T01:10:52.194 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: mon.a@-1(???).mds e1 print_map 2023-06-23T01:10:52.194 INFO:journalctl@ceph.mon.a.smithi120.stdout: e1 2023-06-23T01:10:52.195 INFO:journalctl@ceph.mon.a.smithi120.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-06-23T01:10:52.195 INFO:journalctl@ceph.mon.a.smithi120.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-06-23T01:10:52.195 INFO:journalctl@ceph.mon.a.smithi120.stdout: legacy client fscid: -1 2023-06-23T01:10:52.195 INFO:journalctl@ceph.mon.a.smithi120.stdout: 2023-06-23T01:10:52.195 INFO:journalctl@ceph.mon.a.smithi120.stdout: No filesystems configured 2023-06-23T01:10:52.195 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-06-23T01:10:52.195 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-06-23T01:10:52.195 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-06-23T01:10:52.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-06-23T01:10:52.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-06-23T01:10:52.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: expand_channel_meta expand map: {default=false} 2023-06-23T01:10:52.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: expand_channel_meta from 'false' to 'false' 2023-06-23T01:10:52.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: expand_channel_meta expanded map: {default=false} 2023-06-23T01:10:52.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: expand_channel_meta expand map: {default=info} 2023-06-23T01:10:52.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: expand_channel_meta from 'info' to 'info' 2023-06-23T01:10:52.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: expand_channel_meta expanded map: {default=info} 2023-06-23T01:10:52.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: expand_channel_meta expand map: {default=daemon} 2023-06-23T01:10:52.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: expand_channel_meta from 'daemon' to 'daemon' 2023-06-23T01:10:52.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: expand_channel_meta expanded map: {default=daemon} 2023-06-23T01:10:52.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: expand_channel_meta expand map: {default=debug} 2023-06-23T01:10:52.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: expand_channel_meta from 'debug' to 'debug' 2023-06-23T01:10:52.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: expand_channel_meta expanded map: {default=debug} 2023-06-23T01:10:52.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 bash[79749]: 06b7856cc884f5a70a1d659cc47b369068a41a113c26807c3498fdb00e485f4c 2023-06-23T01:10:52.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 systemd[1]: Started Ceph mon.a for 9efbfa8e-1162-11ee-9b2b-001a4aab830c. 2023-06-23T01:10:52.198 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: mon.a is new leader, mons a in quorum (ranks 0) 2023-06-23T01:10:52.198 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: monmap e1: 1 mons at {a=[v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0]} removed_ranks: {} 2023-06-23T01:10:52.198 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: fsmap 2023-06-23T01:10:52.198 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: osdmap e1: 0 total, 0 up, 0 in 2023-06-23T01:10:52.198 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:51 smithi120 ceph-mon[79813]: mgrmap e1: no daemons active 2023-06-23T01:10:52.880 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.442+0000 7f6f5865c700 1 Processor -- start 2023-06-23T01:10:52.884 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.442+0000 7f6f5865c700 1 -- start start 2023-06-23T01:10:52.884 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.442+0000 7f6f5865c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f50104470 0x7f6f50104880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:10:52.884 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.442+0000 7f6f5865c700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f50104dc0 con 0x7f6f50104470 2023-06-23T01:10:52.884 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.443+0000 7f6f563f8700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f50104470 0x7f6f50104880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:10:52.884 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.443+0000 7f6f563f8700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f50104470 0x7f6f50104880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34200/0 (socket says 172.21.15.120:34200) 2023-06-23T01:10:52.885 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.443+0000 7f6f563f8700 1 -- 172.21.15.120:0/1988717512 learned_addr learned my addr 172.21.15.120:0/1988717512 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:10:52.885 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.443+0000 7f6f563f8700 1 -- 172.21.15.120:0/1988717512 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f50104f00 con 0x7f6f50104470 2023-06-23T01:10:52.885 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.443+0000 7f6f563f8700 1 --2- 172.21.15.120:0/1988717512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f50104470 0x7f6f50104880 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c0098e0 tx=0x7f6f4c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a7e26d156551bf2 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:10:52.885 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.444+0000 7f6f553f6700 1 -- 172.21.15.120:0/1988717512 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6f4c009e80 con 0x7f6f50104470 2023-06-23T01:10:52.885 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.444+0000 7f6f553f6700 1 -- 172.21.15.120:0/1988717512 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6f4c00bd10 con 0x7f6f50104470 2023-06-23T01:10:52.885 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.444+0000 7f6f553f6700 1 -- 172.21.15.120:0/1988717512 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6f4c011ce0 con 0x7f6f50104470 2023-06-23T01:10:52.885 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.444+0000 7f6f5865c700 1 -- 172.21.15.120:0/1988717512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f50104470 msgr2=0x7f6f50104880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:10:52.886 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.444+0000 7f6f5865c700 1 --2- 172.21.15.120:0/1988717512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f50104470 0x7f6f50104880 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c0098e0 tx=0x7f6f4c009250 comp rx=0 tx=0).stop 2023-06-23T01:10:52.886 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.444+0000 7f6f5865c700 1 -- 172.21.15.120:0/1988717512 shutdown_connections 2023-06-23T01:10:52.886 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.444+0000 7f6f5865c700 1 --2- 172.21.15.120:0/1988717512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f50104470 0x7f6f50104880 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:10:52.886 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.444+0000 7f6f5865c700 1 -- 172.21.15.120:0/1988717512 >> 172.21.15.120:0/1988717512 conn(0x7f6f50077a20 msgr2=0x7f6f501023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:10:52.886 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.444+0000 7f6f5865c700 1 -- 172.21.15.120:0/1988717512 shutdown_connections 2023-06-23T01:10:52.886 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.445+0000 7f6f5865c700 1 -- 172.21.15.120:0/1988717512 wait complete. 2023-06-23T01:10:52.886 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.445+0000 7f6f5865c700 1 Processor -- start 2023-06-23T01:10:52.886 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.445+0000 7f6f5865c700 1 -- start start 2023-06-23T01:10:52.887 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.446+0000 7f6f5865c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f50104470 0x7f6f501130e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:10:52.887 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.446+0000 7f6f5865c700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f50113620 con 0x7f6f50104470 2023-06-23T01:10:52.887 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.446+0000 7f6f563f8700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f50104470 0x7f6f501130e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:10:52.887 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.446+0000 7f6f563f8700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f50104470 0x7f6f501130e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34216/0 (socket says 172.21.15.120:34216) 2023-06-23T01:10:52.887 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.446+0000 7f6f563f8700 1 -- 172.21.15.120:0/3932451330 learned_addr learned my addr 172.21.15.120:0/3932451330 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:10:52.887 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.446+0000 7f6f563f8700 1 -- 172.21.15.120:0/3932451330 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f4c0094f0 con 0x7f6f50104470 2023-06-23T01:10:52.887 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.447+0000 7f6f563f8700 1 --2- 172.21.15.120:0/3932451330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f50104470 0x7f6f501130e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c00bce0 tx=0x7f6f4c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:10:52.888 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.447+0000 7f6f46ffd700 1 -- 172.21.15.120:0/3932451330 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6f4c01a640 con 0x7f6f50104470 2023-06-23T01:10:52.888 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.447+0000 7f6f46ffd700 1 -- 172.21.15.120:0/3932451330 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6f4c01ac50 con 0x7f6f50104470 2023-06-23T01:10:52.888 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.447+0000 7f6f5865c700 1 -- 172.21.15.120:0/3932451330 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f50113820 con 0x7f6f50104470 2023-06-23T01:10:52.888 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.447+0000 7f6f46ffd700 1 -- 172.21.15.120:0/3932451330 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6f4c022b20 con 0x7f6f50104470 2023-06-23T01:10:52.888 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.447+0000 7f6f5865c700 1 -- 172.21.15.120:0/3932451330 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f501144c0 con 0x7f6f50104470 2023-06-23T01:10:52.888 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.447+0000 7f6f46ffd700 1 -- 172.21.15.120:0/3932451330 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f6f4c02c440 con 0x7f6f50104470 2023-06-23T01:10:52.888 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.447+0000 7f6f46ffd700 1 -- 172.21.15.120:0/3932451330 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f6f4c02b730 con 0x7f6f50104470 2023-06-23T01:10:52.888 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.448+0000 7f6f5865c700 1 -- 172.21.15.120:0/3932451330 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f5005f060 con 0x7f6f50104470 2023-06-23T01:10:52.889 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.451+0000 7f6f46ffd700 1 -- 172.21.15.120:0/3932451330 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f6f4c01adc0 con 0x7f6f50104470 2023-06-23T01:10:52.889 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.519+0000 7f6f5865c700 1 -- 172.21.15.120:0/3932451330 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f6f501062c0 con 0x7f6f50104470 2023-06-23T01:10:52.889 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.522+0000 7f6f46ffd700 1 -- 172.21.15.120:0/3932451330 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f6f4c022400 con 0x7f6f50104470 2023-06-23T01:10:52.889 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.523+0000 7f6f5865c700 1 -- 172.21.15.120:0/3932451330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f50104470 msgr2=0x7f6f501130e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:10:52.889 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.523+0000 7f6f5865c700 1 --2- 172.21.15.120:0/3932451330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f50104470 0x7f6f501130e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c00bce0 tx=0x7f6f4c009250 comp rx=0 tx=0).stop 2023-06-23T01:10:52.889 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.523+0000 7f6f5865c700 1 -- 172.21.15.120:0/3932451330 shutdown_connections 2023-06-23T01:10:52.889 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.523+0000 7f6f5865c700 1 --2- 172.21.15.120:0/3932451330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f50104470 0x7f6f501130e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:10:52.890 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.523+0000 7f6f5865c700 1 -- 172.21.15.120:0/3932451330 >> 172.21.15.120:0/3932451330 conn(0x7f6f50077a20 msgr2=0x7f6f50101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:10:52.890 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.523+0000 7f6f5865c700 1 -- 172.21.15.120:0/3932451330 shutdown_connections 2023-06-23T01:10:52.890 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:52.523+0000 7f6f5865c700 1 -- 172.21.15.120:0/3932451330 wait complete. 2023-06-23T01:10:52.890 INFO:teuthology.orchestra.run.smithi120.stdout:Wrote config to /etc/ceph/ceph.conf 2023-06-23T01:10:52.891 INFO:teuthology.orchestra.run.smithi120.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-06-23T01:10:52.891 INFO:teuthology.orchestra.run.smithi120.stdout:Creating mgr... 2023-06-23T01:10:52.891 INFO:teuthology.orchestra.run.smithi120.stdout:Verifying port 9283 ... 2023-06-23T01:10:53.165 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mgr.x 2023-06-23T01:10:53.165 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mgr.x.service: Unit ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mgr.x.service not loaded. 2023-06-23T01:10:53.361 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c.target.wants/ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mgr.x.service → /etc/systemd/system/ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@.service. 2023-06-23T01:10:53.510 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:10:53 smithi120 systemd[1]: Starting Ceph mgr.x for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:10:54.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:53 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/3932451330' entity='client.admin' 2023-06-23T01:10:54.245 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-06-23T01:10:54.246 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout disabled 2023-06-23T01:10:54.262 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-06-23T01:10:54.263 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout inactive 2023-06-23T01:10:54.263 INFO:teuthology.orchestra.run.smithi120.stdout:firewalld.service is not enabled 2023-06-23T01:10:54.263 INFO:teuthology.orchestra.run.smithi120.stdout:Not possible to enable service . firewalld.service is not available 2023-06-23T01:10:54.280 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-06-23T01:10:54.280 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout disabled 2023-06-23T01:10:54.297 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-06-23T01:10:54.298 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout inactive 2023-06-23T01:10:54.298 INFO:teuthology.orchestra.run.smithi120.stdout:firewalld.service is not enabled 2023-06-23T01:10:54.298 INFO:teuthology.orchestra.run.smithi120.stdout:Not possible to open ports <[9283]>. firewalld.service is not available 2023-06-23T01:10:54.298 INFO:teuthology.orchestra.run.smithi120.stdout:Waiting for mgr to start... 2023-06-23T01:10:54.298 INFO:teuthology.orchestra.run.smithi120.stdout:Waiting for mgr... 2023-06-23T01:10:54.510 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:10:54 smithi120 bash[80184]: 9aa3cde57b04acedcd092ab81fa8186cebf4da63cac1f5570642eba4e8026044 2023-06-23T01:10:54.510 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:10:54 smithi120 systemd[1]: Started Ceph mgr.x for 9efbfa8e-1162-11ee-9b2b-001a4aab830c. 2023-06-23T01:10:55.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:55 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/1350359951' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-06-23T01:10:55.376 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 2023-06-23T01:10:55.377 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout { 2023-06-23T01:10:55.377 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "fsid": "9efbfa8e-1162-11ee-9b2b-001a4aab830c", 2023-06-23T01:10:55.377 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "health": { 2023-06-23T01:10:55.378 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-06-23T01:10:55.378 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-06-23T01:10:55.378 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-06-23T01:10:55.378 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2023-06-23T01:10:55.379 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-06-23T01:10:55.379 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-06-23T01:10:55.379 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 0 2023-06-23T01:10:55.379 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout ], 2023-06-23T01:10:55.380 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-06-23T01:10:55.380 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "a" 2023-06-23T01:10:55.380 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout ], 2023-06-23T01:10:55.380 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2023-06-23T01:10:55.381 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "monmap": { 2023-06-23T01:10:55.381 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-23T01:10:55.381 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2023-06-23T01:10:55.382 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-06-23T01:10:55.382 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2023-06-23T01:10:55.382 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-06-23T01:10:55.382 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-23T01:10:55.383 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-06-23T01:10:55.383 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-06-23T01:10:55.383 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-06-23T01:10:55.392 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-06-23T01:10:55.392 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-06-23T01:10:55.392 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-06-23T01:10:55.392 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2023-06-23T01:10:55.393 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-06-23T01:10:55.393 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-06-23T01:10:55.393 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-06-23T01:10:55.393 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-06-23T01:10:55.393 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-06-23T01:10:55.393 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-06-23T01:10:55.393 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-06-23T01:10:55.394 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-06-23T01:10:55.394 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-06-23T01:10:55.394 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2023-06-23T01:10:55.394 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-06-23T01:10:55.394 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-23T01:10:55.394 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-06-23T01:10:55.394 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-06-23T01:10:55.394 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2023-06-23T01:10:55.395 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-06-23T01:10:55.395 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "available": false, 2023-06-23T01:10:55.395 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-06-23T01:10:55.395 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "modules": [ 2023-06-23T01:10:55.395 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "iostat", 2023-06-23T01:10:55.395 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "nfs", 2023-06-23T01:10:55.395 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "restful" 2023-06-23T01:10:55.396 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout ], 2023-06-23T01:10:55.396 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "services": {} 2023-06-23T01:10:55.396 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2023-06-23T01:10:55.396 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-06-23T01:10:55.396 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-23T01:10:55.396 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "modified": "2023-06-23T01:10:46.841001+0000", 2023-06-23T01:10:55.396 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "services": {} 2023-06-23T01:10:55.397 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2023-06-23T01:10:55.397 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-06-23T01:10:55.397 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout } 2023-06-23T01:10:55.397 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.858+0000 7f49ea423700 1 Processor -- start 2023-06-23T01:10:55.397 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.858+0000 7f49ea423700 1 -- start start 2023-06-23T01:10:55.397 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.858+0000 7f49ea423700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49e4102210 0x7f49e4102620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:10:55.398 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.858+0000 7f49ea423700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e4102b60 con 0x7f49e4102210 2023-06-23T01:10:55.398 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.859+0000 7f49e3fff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49e4102210 0x7f49e4102620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:10:55.398 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.859+0000 7f49e3fff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49e4102210 0x7f49e4102620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34252/0 (socket says 172.21.15.120:34252) 2023-06-23T01:10:55.398 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.859+0000 7f49e3fff700 1 -- 172.21.15.120:0/2019996735 learned_addr learned my addr 172.21.15.120:0/2019996735 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:10:55.398 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.859+0000 7f49e3fff700 1 -- 172.21.15.120:0/2019996735 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e4102ca0 con 0x7f49e4102210 2023-06-23T01:10:55.399 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.859+0000 7f49e3fff700 1 --2- 172.21.15.120:0/2019996735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49e4102210 0x7f49e4102620 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f49d40098e0 tx=0x7f49d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa8ca61345a6769f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:10:55.399 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.860+0000 7f49e2ffd700 1 -- 172.21.15.120:0/2019996735 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49d4009e80 con 0x7f49e4102210 2023-06-23T01:10:55.399 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.860+0000 7f49e2ffd700 1 -- 172.21.15.120:0/2019996735 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f49d400bd10 con 0x7f49e4102210 2023-06-23T01:10:55.399 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.860+0000 7f49e2ffd700 1 -- 172.21.15.120:0/2019996735 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49d4011ce0 con 0x7f49e4102210 2023-06-23T01:10:55.399 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.860+0000 7f49ea423700 1 -- 172.21.15.120:0/2019996735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49e4102210 msgr2=0x7f49e4102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:10:55.399 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.860+0000 7f49ea423700 1 --2- 172.21.15.120:0/2019996735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49e4102210 0x7f49e4102620 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f49d40098e0 tx=0x7f49d4009250 comp rx=0 tx=0).stop 2023-06-23T01:10:55.399 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.860+0000 7f49ea423700 1 -- 172.21.15.120:0/2019996735 shutdown_connections 2023-06-23T01:10:55.399 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.860+0000 7f49ea423700 1 --2- 172.21.15.120:0/2019996735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49e4102210 0x7f49e4102620 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:10:55.400 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.860+0000 7f49ea423700 1 -- 172.21.15.120:0/2019996735 >> 172.21.15.120:0/2019996735 conn(0x7f49e40fd990 msgr2=0x7f49e40ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:10:55.400 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.860+0000 7f49ea423700 1 -- 172.21.15.120:0/2019996735 shutdown_connections 2023-06-23T01:10:55.400 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.860+0000 7f49ea423700 1 -- 172.21.15.120:0/2019996735 wait complete. 2023-06-23T01:10:55.400 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.861+0000 7f49ea423700 1 Processor -- start 2023-06-23T01:10:55.400 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.861+0000 7f49ea423700 1 -- start start 2023-06-23T01:10:55.401 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.862+0000 7f49ea423700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49e4102210 0x7f49e4110ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:10:55.401 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.862+0000 7f49ea423700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e4111420 con 0x7f49e4102210 2023-06-23T01:10:55.401 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.862+0000 7f49e3fff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49e4102210 0x7f49e4110ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:10:55.401 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.862+0000 7f49e3fff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49e4102210 0x7f49e4110ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34258/0 (socket says 172.21.15.120:34258) 2023-06-23T01:10:55.401 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.862+0000 7f49e3fff700 1 -- 172.21.15.120:0/1350359951 learned_addr learned my addr 172.21.15.120:0/1350359951 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:10:55.401 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.863+0000 7f49e3fff700 1 -- 172.21.15.120:0/1350359951 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d40094f0 con 0x7f49e4102210 2023-06-23T01:10:55.401 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.863+0000 7f49e3fff700 1 --2- 172.21.15.120:0/1350359951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49e4102210 0x7f49e4110ee0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f49d4006b00 tx=0x7f49d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:10:55.402 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.863+0000 7f49e0ff9700 1 -- 172.21.15.120:0/1350359951 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49d401a640 con 0x7f49e4102210 2023-06-23T01:10:55.402 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.863+0000 7f49ea423700 1 -- 172.21.15.120:0/1350359951 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e4111620 con 0x7f49e4102210 2023-06-23T01:10:55.402 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.863+0000 7f49e0ff9700 1 -- 172.21.15.120:0/1350359951 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f49d401ac50 con 0x7f49e4102210 2023-06-23T01:10:55.402 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.863+0000 7f49e0ff9700 1 -- 172.21.15.120:0/1350359951 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49d4022bb0 con 0x7f49e4102210 2023-06-23T01:10:55.402 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.863+0000 7f49ea423700 1 -- 172.21.15.120:0/1350359951 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e41122c0 con 0x7f49e4102210 2023-06-23T01:10:55.402 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.863+0000 7f49e0ff9700 1 -- 172.21.15.120:0/1350359951 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f49d402c440 con 0x7f49e4102210 2023-06-23T01:10:55.402 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.863+0000 7f49e0ff9700 1 -- 172.21.15.120:0/1350359951 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f49d402b730 con 0x7f49e4102210 2023-06-23T01:10:55.403 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.864+0000 7f49ea423700 1 -- 172.21.15.120:0/1350359951 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49e405f060 con 0x7f49e4102210 2023-06-23T01:10:55.403 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.867+0000 7f49e0ff9700 1 -- 172.21.15.120:0/1350359951 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f49d401adc0 con 0x7f49e4102210 2023-06-23T01:10:55.403 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.963+0000 7f49ea423700 1 -- 172.21.15.120:0/1350359951 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f49e41040d0 con 0x7f49e4102210 2023-06-23T01:10:55.404 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.963+0000 7f49e0ff9700 1 -- 172.21.15.120:0/1350359951 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7f49d4022490 con 0x7f49e4102210 2023-06-23T01:10:55.404 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.965+0000 7f49ea423700 1 -- 172.21.15.120:0/1350359951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49e4102210 msgr2=0x7f49e4110ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:10:55.404 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.965+0000 7f49ea423700 1 --2- 172.21.15.120:0/1350359951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49e4102210 0x7f49e4110ee0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f49d4006b00 tx=0x7f49d4009250 comp rx=0 tx=0).stop 2023-06-23T01:10:55.404 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.965+0000 7f49ea423700 1 -- 172.21.15.120:0/1350359951 shutdown_connections 2023-06-23T01:10:55.405 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.965+0000 7f49ea423700 1 --2- 172.21.15.120:0/1350359951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49e4102210 0x7f49e4110ee0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:10:55.405 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.965+0000 7f49ea423700 1 -- 172.21.15.120:0/1350359951 >> 172.21.15.120:0/1350359951 conn(0x7f49e40fd990 msgr2=0x7f49e40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:10:55.405 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.965+0000 7f49ea423700 1 -- 172.21.15.120:0/1350359951 shutdown_connections 2023-06-23T01:10:55.405 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:54.965+0000 7f49ea423700 1 -- 172.21.15.120:0/1350359951 wait complete. 2023-06-23T01:10:55.406 INFO:teuthology.orchestra.run.smithi120.stdout:mgr not available, waiting (1/15)... 2023-06-23T01:10:56.092 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:10:55 smithi120 conmon[80224]: 2023-06-23T01:10:55.771+0000 7f7c6adc31c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-06-23T01:10:56.092 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:10:55 smithi120 conmon[80224]: 2023-06-23T01:10:55.975+0000 7f7c6adc31c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-06-23T01:10:56.092 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:10:56 smithi120 conmon[80224]: 2023-06-23T01:10:56.089+0000 7f7c6adc31c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-06-23T01:10:56.607 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:10:56 smithi120 conmon[80224]: 2023-06-23T01:10:56.353+0000 7f7c6adc31c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-06-23T01:10:56.901 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:10:56 smithi120 conmon[80224]: 2023-06-23T01:10:56.603+0000 7f7c6adc31c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-06-23T01:10:56.902 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:10:56 smithi120 conmon[80224]: 2023-06-23T01:10:56.796+0000 7f7c6adc31c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-06-23T01:10:56.902 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:10:56 smithi120 conmon[80224]: 2023-06-23T01:10:56.898+0000 7f7c6adc31c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-06-23T01:10:58.010 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:10:57 smithi120 conmon[80224]: 2023-06-23T01:10:57.691+0000 7f7c6adc31c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-06-23T01:10:58.347 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:10:58 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/949100102' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-06-23T01:10:58.473 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 2023-06-23T01:10:58.474 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout { 2023-06-23T01:10:58.474 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "fsid": "9efbfa8e-1162-11ee-9b2b-001a4aab830c", 2023-06-23T01:10:58.474 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "health": { 2023-06-23T01:10:58.474 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-06-23T01:10:58.474 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-06-23T01:10:58.475 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-06-23T01:10:58.475 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2023-06-23T01:10:58.475 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-06-23T01:10:58.475 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-06-23T01:10:58.475 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 0 2023-06-23T01:10:58.475 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout ], 2023-06-23T01:10:58.475 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-06-23T01:10:58.476 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "a" 2023-06-23T01:10:58.476 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout ], 2023-06-23T01:10:58.476 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2023-06-23T01:10:58.476 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "monmap": { 2023-06-23T01:10:58.476 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-23T01:10:58.476 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2023-06-23T01:10:58.477 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-06-23T01:10:58.477 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2023-06-23T01:10:58.477 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-06-23T01:10:58.477 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-23T01:10:58.477 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-06-23T01:10:58.477 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-06-23T01:10:58.478 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-06-23T01:10:58.478 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-06-23T01:10:58.478 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-06-23T01:10:58.478 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-06-23T01:10:58.478 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2023-06-23T01:10:58.478 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-06-23T01:10:58.478 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-06-23T01:10:58.479 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-06-23T01:10:58.479 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-06-23T01:10:58.479 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-06-23T01:10:58.479 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-06-23T01:10:58.479 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-06-23T01:10:58.479 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-06-23T01:10:58.479 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-06-23T01:10:58.479 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2023-06-23T01:10:58.480 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-06-23T01:10:58.480 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-23T01:10:58.480 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-06-23T01:10:58.480 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-06-23T01:10:58.481 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2023-06-23T01:10:58.481 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-06-23T01:10:58.481 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "available": false, 2023-06-23T01:10:58.481 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-06-23T01:10:58.481 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "modules": [ 2023-06-23T01:10:58.481 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "iostat", 2023-06-23T01:10:58.482 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "nfs", 2023-06-23T01:10:58.482 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "restful" 2023-06-23T01:10:58.482 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout ], 2023-06-23T01:10:58.482 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "services": {} 2023-06-23T01:10:58.482 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2023-06-23T01:10:58.482 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-06-23T01:10:58.482 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-23T01:10:58.482 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "modified": "2023-06-23T01:10:46.841001+0000", 2023-06-23T01:10:58.483 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "services": {} 2023-06-23T01:10:58.483 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2023-06-23T01:10:58.483 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-06-23T01:10:58.483 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout } 2023-06-23T01:10:58.483 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:57.962+0000 7ff53cc26700 1 Processor -- start 2023-06-23T01:10:58.483 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:57.962+0000 7ff53cc26700 1 -- start start 2023-06-23T01:10:58.483 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:57.963+0000 7ff53cc26700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff538102dd0 0x7ff5381051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:10:58.484 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:57.963+0000 7ff53cc26700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff538070ee0 con 0x7ff538102dd0 2023-06-23T01:10:58.484 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:57.963+0000 7ff53659c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff538102dd0 0x7ff5381051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:10:58.484 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:57.963+0000 7ff53659c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff538102dd0 0x7ff5381051b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34274/0 (socket says 172.21.15.120:34274) 2023-06-23T01:10:58.484 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:57.963+0000 7ff53659c700 1 -- 172.21.15.120:0/1497656505 learned_addr learned my addr 172.21.15.120:0/1497656505 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:10:58.484 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:57.963+0000 7ff53659c700 1 -- 172.21.15.120:0/1497656505 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff538071020 con 0x7ff538102dd0 2023-06-23T01:10:58.484 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:57.964+0000 7ff53659c700 1 --2- 172.21.15.120:0/1497656505 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff538102dd0 0x7ff5381051b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff5280098e0 tx=0x7ff528009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83bf4eb3fa952eb7 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:10:58.484 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:57.964+0000 7ff53559a700 1 -- 172.21.15.120:0/1497656505 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff528009e80 con 0x7ff538102dd0 2023-06-23T01:10:58.485 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:57.964+0000 7ff53559a700 1 -- 172.21.15.120:0/1497656505 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff52800bd10 con 0x7ff538102dd0 2023-06-23T01:10:58.485 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:57.964+0000 7ff53559a700 1 -- 172.21.15.120:0/1497656505 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff528011ce0 con 0x7ff538102dd0 2023-06-23T01:10:58.485 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:57.965+0000 7ff53cc26700 1 -- 172.21.15.120:0/1497656505 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff538102dd0 msgr2=0x7ff5381051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:10:58.485 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:57.965+0000 7ff53cc26700 1 --2- 172.21.15.120:0/1497656505 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff538102dd0 0x7ff5381051b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff5280098e0 tx=0x7ff528009250 comp rx=0 tx=0).stop 2023-06-23T01:10:58.485 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:57.965+0000 7ff53cc26700 1 -- 172.21.15.120:0/1497656505 shutdown_connections 2023-06-23T01:10:58.485 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:57.965+0000 7ff53cc26700 1 --2- 172.21.15.120:0/1497656505 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff538102dd0 0x7ff5381051b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:10:58.485 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:57.965+0000 7ff53cc26700 1 -- 172.21.15.120:0/1497656505 >> 172.21.15.120:0/1497656505 conn(0x7ff5380fc890 msgr2=0x7ff5380feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:10:58.486 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:57.965+0000 7ff53cc26700 1 -- 172.21.15.120:0/1497656505 shutdown_connections 2023-06-23T01:10:58.486 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:57.965+0000 7ff53cc26700 1 -- 172.21.15.120:0/1497656505 wait complete. 2023-06-23T01:10:58.486 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:57.966+0000 7ff53cc26700 1 Processor -- start 2023-06-23T01:10:58.486 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:57.966+0000 7ff53cc26700 1 -- start start 2023-06-23T01:10:58.486 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:57.966+0000 7ff53cc26700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff538102dd0 0x7ff538110e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:10:58.486 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:57.966+0000 7ff53cc26700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5381113d0 con 0x7ff538102dd0 2023-06-23T01:10:58.486 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:57.966+0000 7ff53659c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff538102dd0 0x7ff538110e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:10:58.486 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:57.966+0000 7ff53659c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff538102dd0 0x7ff538110e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34290/0 (socket says 172.21.15.120:34290) 2023-06-23T01:10:58.487 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:57.966+0000 7ff53659c700 1 -- 172.21.15.120:0/949100102 learned_addr learned my addr 172.21.15.120:0/949100102 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:10:58.487 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:57.967+0000 7ff53659c700 1 -- 172.21.15.120:0/949100102 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5280094f0 con 0x7ff538102dd0 2023-06-23T01:10:58.487 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:57.967+0000 7ff53659c700 1 --2- 172.21.15.120:0/949100102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff538102dd0 0x7ff538110e90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff528011970 tx=0x7ff528009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:10:58.487 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:57.967+0000 7ff526ffd700 1 -- 172.21.15.120:0/949100102 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff528009e80 con 0x7ff538102dd0 2023-06-23T01:10:58.487 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:57.967+0000 7ff526ffd700 1 -- 172.21.15.120:0/949100102 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff52801aaf0 con 0x7ff538102dd0 2023-06-23T01:10:58.487 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:57.967+0000 7ff53cc26700 1 -- 172.21.15.120:0/949100102 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff538111630 con 0x7ff538102dd0 2023-06-23T01:10:58.487 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:57.967+0000 7ff526ffd700 1 -- 172.21.15.120:0/949100102 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff528022a90 con 0x7ff538102dd0 2023-06-23T01:10:58.488 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:57.968+0000 7ff53cc26700 1 -- 172.21.15.120:0/949100102 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff538112270 con 0x7ff538102dd0 2023-06-23T01:10:58.488 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:57.968+0000 7ff526ffd700 1 -- 172.21.15.120:0/949100102 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7ff528022c90 con 0x7ff538102dd0 2023-06-23T01:10:58.488 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:57.968+0000 7ff526ffd700 1 -- 172.21.15.120:0/949100102 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7ff52801a640 con 0x7ff538102dd0 2023-06-23T01:10:58.488 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:57.969+0000 7ff53cc26700 1 -- 172.21.15.120:0/949100102 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff53805f060 con 0x7ff538102dd0 2023-06-23T01:10:58.488 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:57.971+0000 7ff526ffd700 1 -- 172.21.15.120:0/949100102 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7ff52801a800 con 0x7ff538102dd0 2023-06-23T01:10:58.488 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:58.046+0000 7ff53cc26700 1 -- 172.21.15.120:0/949100102 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7ff538106290 con 0x7ff538102dd0 2023-06-23T01:10:58.488 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:58.046+0000 7ff526ffd700 1 -- 172.21.15.120:0/949100102 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7ff528022400 con 0x7ff538102dd0 2023-06-23T01:10:58.489 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:58.047+0000 7ff53cc26700 1 -- 172.21.15.120:0/949100102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff538102dd0 msgr2=0x7ff538110e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:10:58.489 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:58.047+0000 7ff53cc26700 1 --2- 172.21.15.120:0/949100102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff538102dd0 0x7ff538110e90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff528011970 tx=0x7ff528009250 comp rx=0 tx=0).stop 2023-06-23T01:10:58.489 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:58.047+0000 7ff53cc26700 1 -- 172.21.15.120:0/949100102 shutdown_connections 2023-06-23T01:10:58.489 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:58.047+0000 7ff53cc26700 1 --2- 172.21.15.120:0/949100102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff538102dd0 0x7ff538110e90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:10:58.489 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:58.047+0000 7ff53cc26700 1 -- 172.21.15.120:0/949100102 >> 172.21.15.120:0/949100102 conn(0x7ff5380fc890 msgr2=0x7ff5380fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:10:58.489 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:58.048+0000 7ff53cc26700 1 -- 172.21.15.120:0/949100102 shutdown_connections 2023-06-23T01:10:58.489 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:10:58.048+0000 7ff53cc26700 1 -- 172.21.15.120:0/949100102 wait complete. 2023-06-23T01:10:58.490 INFO:teuthology.orchestra.run.smithi120.stdout:mgr not available, waiting (2/15)... 2023-06-23T01:10:58.760 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:10:58 smithi120 conmon[80224]: 2023-06-23T01:10:58.343+0000 7f7c6adc31c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-06-23T01:10:58.760 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:10:58 smithi120 conmon[80224]: 2023-06-23T01:10:58.470+0000 7f7c6adc31c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-06-23T01:10:58.760 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:10:58 smithi120 conmon[80224]: 2023-06-23T01:10:58.589+0000 7f7c6adc31c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-06-23T01:10:59.686 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:10:59 smithi120 conmon[80224]: 2023-06-23T01:10:59.354+0000 7f7c6adc31c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-06-23T01:10:59.686 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:10:59 smithi120 conmon[80224]: 2023-06-23T01:10:59.569+0000 7f7c6adc31c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-06-23T01:10:59.968 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:10:59 smithi120 conmon[80224]: 2023-06-23T01:10:59.683+0000 7f7c6adc31c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-06-23T01:10:59.969 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:10:59 smithi120 conmon[80224]: 2023-06-23T01:10:59.848+0000 7f7c6adc31c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-06-23T01:10:59.969 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:10:59 smithi120 conmon[80224]: 2023-06-23T01:10:59.965+0000 7f7c6adc31c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-06-23T01:11:01.437 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 2023-06-23T01:11:01.439 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout { 2023-06-23T01:11:01.439 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "fsid": "9efbfa8e-1162-11ee-9b2b-001a4aab830c", 2023-06-23T01:11:01.439 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "health": { 2023-06-23T01:11:01.439 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-06-23T01:11:01.439 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-06-23T01:11:01.440 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-06-23T01:11:01.440 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2023-06-23T01:11:01.440 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-06-23T01:11:01.440 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-06-23T01:11:01.440 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 0 2023-06-23T01:11:01.440 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout ], 2023-06-23T01:11:01.440 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-06-23T01:11:01.441 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "a" 2023-06-23T01:11:01.441 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout ], 2023-06-23T01:11:01.441 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2023-06-23T01:11:01.441 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "monmap": { 2023-06-23T01:11:01.441 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-23T01:11:01.441 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2023-06-23T01:11:01.441 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-06-23T01:11:01.441 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2023-06-23T01:11:01.442 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-06-23T01:11:01.442 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-23T01:11:01.442 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-06-23T01:11:01.442 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-06-23T01:11:01.442 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-06-23T01:11:01.447 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-06-23T01:11:01.448 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-06-23T01:11:01.448 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-06-23T01:11:01.448 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2023-06-23T01:11:01.448 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-06-23T01:11:01.448 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-06-23T01:11:01.448 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-06-23T01:11:01.448 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-06-23T01:11:01.449 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-06-23T01:11:01.449 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-06-23T01:11:01.449 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-06-23T01:11:01.449 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-06-23T01:11:01.449 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-06-23T01:11:01.449 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2023-06-23T01:11:01.449 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-06-23T01:11:01.450 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-23T01:11:01.450 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-06-23T01:11:01.450 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-06-23T01:11:01.450 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2023-06-23T01:11:01.450 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-06-23T01:11:01.450 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "available": false, 2023-06-23T01:11:01.450 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-06-23T01:11:01.450 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "modules": [ 2023-06-23T01:11:01.451 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "iostat", 2023-06-23T01:11:01.451 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "nfs", 2023-06-23T01:11:01.451 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "restful" 2023-06-23T01:11:01.451 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout ], 2023-06-23T01:11:01.451 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "services": {} 2023-06-23T01:11:01.451 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2023-06-23T01:11:01.451 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-06-23T01:11:01.451 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-23T01:11:01.452 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "modified": "2023-06-23T01:10:46.841001+0000", 2023-06-23T01:11:01.452 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "services": {} 2023-06-23T01:11:01.452 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2023-06-23T01:11:01.452 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-06-23T01:11:01.452 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout } 2023-06-23T01:11:01.452 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:00.984+0000 7fb7dcac1700 1 Processor -- start 2023-06-23T01:11:01.452 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:00.984+0000 7fb7dcac1700 1 -- start start 2023-06-23T01:11:01.452 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:00.985+0000 7fb7dcac1700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7d8077d10 0x7fb7d80761b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:01.453 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:00.985+0000 7fb7dcac1700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d80766f0 con 0x7fb7d8077d10 2023-06-23T01:11:01.453 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:00.985+0000 7fb7d659c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7d8077d10 0x7fb7d80761b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:01.453 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:00.985+0000 7fb7d659c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7d8077d10 0x7fb7d80761b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34302/0 (socket says 172.21.15.120:34302) 2023-06-23T01:11:01.453 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:00.985+0000 7fb7d659c700 1 -- 172.21.15.120:0/2644388230 learned_addr learned my addr 172.21.15.120:0/2644388230 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:01.453 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:00.985+0000 7fb7d659c700 1 -- 172.21.15.120:0/2644388230 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d8076830 con 0x7fb7d8077d10 2023-06-23T01:11:01.453 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:00.986+0000 7fb7d659c700 1 --2- 172.21.15.120:0/2644388230 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7d8077d10 0x7fb7d80761b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb7c80098e0 tx=0x7fb7c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ac31b66136c812e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:01.453 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:00.986+0000 7fb7d559a700 1 -- 172.21.15.120:0/2644388230 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb7c8009e80 con 0x7fb7d8077d10 2023-06-23T01:11:01.454 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:00.986+0000 7fb7d559a700 1 -- 172.21.15.120:0/2644388230 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb7c800bd10 con 0x7fb7d8077d10 2023-06-23T01:11:01.454 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:00.986+0000 7fb7d559a700 1 -- 172.21.15.120:0/2644388230 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb7c8011ce0 con 0x7fb7d8077d10 2023-06-23T01:11:01.454 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:00.987+0000 7fb7dcac1700 1 -- 172.21.15.120:0/2644388230 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7d8077d10 msgr2=0x7fb7d80761b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:01.454 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:00.987+0000 7fb7dcac1700 1 --2- 172.21.15.120:0/2644388230 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7d8077d10 0x7fb7d80761b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb7c80098e0 tx=0x7fb7c8009250 comp rx=0 tx=0).stop 2023-06-23T01:11:01.454 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:00.987+0000 7fb7dcac1700 1 -- 172.21.15.120:0/2644388230 shutdown_connections 2023-06-23T01:11:01.454 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:00.987+0000 7fb7dcac1700 1 --2- 172.21.15.120:0/2644388230 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7d8077d10 0x7fb7d80761b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:01.454 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:00.987+0000 7fb7dcac1700 1 -- 172.21.15.120:0/2644388230 >> 172.21.15.120:0/2644388230 conn(0x7fb7d80fd990 msgr2=0x7fb7d80ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:01.454 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:00.987+0000 7fb7dcac1700 1 -- 172.21.15.120:0/2644388230 shutdown_connections 2023-06-23T01:11:01.455 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:00.987+0000 7fb7dcac1700 1 -- 172.21.15.120:0/2644388230 wait complete. 2023-06-23T01:11:01.455 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:00.988+0000 7fb7dcac1700 1 Processor -- start 2023-06-23T01:11:01.455 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:00.988+0000 7fb7dcac1700 1 -- start start 2023-06-23T01:11:01.455 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:00.989+0000 7fb7dcac1700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7d8077d10 0x7fb7d8117530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:01.455 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:00.989+0000 7fb7dcac1700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d8117a70 con 0x7fb7d8077d10 2023-06-23T01:11:01.455 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:00.989+0000 7fb7d659c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7d8077d10 0x7fb7d8117530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:01.455 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:00.989+0000 7fb7d659c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7d8077d10 0x7fb7d8117530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34316/0 (socket says 172.21.15.120:34316) 2023-06-23T01:11:01.455 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:00.989+0000 7fb7d659c700 1 -- 172.21.15.120:0/3110025497 learned_addr learned my addr 172.21.15.120:0/3110025497 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:01.456 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:00.989+0000 7fb7d659c700 1 -- 172.21.15.120:0/3110025497 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c80094f0 con 0x7fb7d8077d10 2023-06-23T01:11:01.456 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:00.989+0000 7fb7d659c700 1 --2- 172.21.15.120:0/3110025497 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7d8077d10 0x7fb7d8117530 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8011970 tx=0x7fb7c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:01.456 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:00.990+0000 7fb7c6ffd700 1 -- 172.21.15.120:0/3110025497 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb7c800be80 con 0x7fb7d8077d10 2023-06-23T01:11:01.456 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:00.990+0000 7fb7c6ffd700 1 -- 172.21.15.120:0/3110025497 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb7c801aaf0 con 0x7fb7d8077d10 2023-06-23T01:11:01.456 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:00.990+0000 7fb7c6ffd700 1 -- 172.21.15.120:0/3110025497 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb7c8022a80 con 0x7fb7d8077d10 2023-06-23T01:11:01.456 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:00.990+0000 7fb7dcac1700 1 -- 172.21.15.120:0/3110025497 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7d8117c70 con 0x7fb7d8077d10 2023-06-23T01:11:01.456 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:00.990+0000 7fb7dcac1700 1 -- 172.21.15.120:0/3110025497 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d8118910 con 0x7fb7d8077d10 2023-06-23T01:11:01.457 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:00.990+0000 7fb7c6ffd700 1 -- 172.21.15.120:0/3110025497 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fb7c8022c70 con 0x7fb7d8077d10 2023-06-23T01:11:01.457 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:00.990+0000 7fb7c6ffd700 1 -- 172.21.15.120:0/3110025497 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fb7c802b3e0 con 0x7fb7d8077d10 2023-06-23T01:11:01.457 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:00.991+0000 7fb7dcac1700 1 -- 172.21.15.120:0/3110025497 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7d805f060 con 0x7fb7d8077d10 2023-06-23T01:11:01.457 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:00.993+0000 7fb7c6ffd700 1 -- 172.21.15.120:0/3110025497 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7fb7c8022400 con 0x7fb7d8077d10 2023-06-23T01:11:01.457 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:01.068+0000 7fb7dcac1700 1 -- 172.21.15.120:0/3110025497 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb7d8104150 con 0x7fb7d8077d10 2023-06-23T01:11:01.457 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:01.068+0000 7fb7c6ffd700 1 -- 172.21.15.120:0/3110025497 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7fb7c801ac60 con 0x7fb7d8077d10 2023-06-23T01:11:01.457 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:01.069+0000 7fb7dcac1700 1 -- 172.21.15.120:0/3110025497 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7d8077d10 msgr2=0x7fb7d8117530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:01.457 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:01.069+0000 7fb7dcac1700 1 --2- 172.21.15.120:0/3110025497 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7d8077d10 0x7fb7d8117530 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8011970 tx=0x7fb7c8009250 comp rx=0 tx=0).stop 2023-06-23T01:11:01.458 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:01.069+0000 7fb7dcac1700 1 -- 172.21.15.120:0/3110025497 shutdown_connections 2023-06-23T01:11:01.458 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:01.069+0000 7fb7dcac1700 1 --2- 172.21.15.120:0/3110025497 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7d8077d10 0x7fb7d8117530 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:01.458 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:01.069+0000 7fb7dcac1700 1 -- 172.21.15.120:0/3110025497 >> 172.21.15.120:0/3110025497 conn(0x7fb7d80fd990 msgr2=0x7fb7d8103520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:01.458 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:01.070+0000 7fb7dcac1700 1 -- 172.21.15.120:0/3110025497 shutdown_connections 2023-06-23T01:11:01.458 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:01.070+0000 7fb7dcac1700 1 -- 172.21.15.120:0/3110025497 wait complete. 2023-06-23T01:11:01.458 INFO:teuthology.orchestra.run.smithi120.stdout:mgr not available, waiting (3/15)... 2023-06-23T01:11:01.499 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:01 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/3110025497' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-06-23T01:11:01.500 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:01 smithi120 conmon[80224]: 2023-06-23T01:11:01.081+0000 7f7c6adc31c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-06-23T01:11:01.500 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:01 smithi120 conmon[80224]: 2023-06-23T01:11:01.203+0000 7f7c6adc31c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-06-23T01:11:01.500 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:01 smithi120 conmon[80224]: 2023-06-23T01:11:01.322+0000 7f7c6adc31c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-06-23T01:11:01.760 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:01 smithi120 conmon[80224]: 2023-06-23T01:11:01.519+0000 7f7c6adc31c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-06-23T01:11:01.760 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:01 smithi120 conmon[80224]: 2023-06-23T01:11:01.644+0000 7f7c6adc31c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-06-23T01:11:02.461 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:02 smithi120 conmon[80224]: 2023-06-23T01:11:02.115+0000 7f7c6adc31c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-06-23T01:11:02.461 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:02 smithi120 conmon[80224]: 2023-06-23T01:11:02.229+0000 7f7c6adc31c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-06-23T01:11:02.461 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:02 smithi120 conmon[80224]: 2023-06-23T01:11:02.343+0000 7f7c6adc31c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-06-23T01:11:02.462 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:02 smithi120 conmon[80224]: 2023-06-23T01:11:02.457+0000 7f7c6adc31c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-06-23T01:11:03.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:02 smithi120 ceph-mon[79813]: Activating manager daemon x 2023-06-23T01:11:03.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:02 smithi120 ceph-mon[79813]: mgrmap e2: x(active, starting, since 0.00369648s) 2023-06-23T01:11:03.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:02 smithi120 ceph-mon[79813]: from='mgr.14100 172.21.15.120:0/3081679864' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-06-23T01:11:03.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:02 smithi120 ceph-mon[79813]: from='mgr.14100 172.21.15.120:0/3081679864' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-06-23T01:11:03.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:02 smithi120 ceph-mon[79813]: from='mgr.14100 172.21.15.120:0/3081679864' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-06-23T01:11:03.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:02 smithi120 ceph-mon[79813]: from='mgr.14100 172.21.15.120:0/3081679864' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-23T01:11:03.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:02 smithi120 ceph-mon[79813]: from='mgr.14100 172.21.15.120:0/3081679864' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-06-23T01:11:03.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:02 smithi120 ceph-mon[79813]: Manager daemon x is now available 2023-06-23T01:11:03.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:02 smithi120 ceph-mon[79813]: from='mgr.14100 172.21.15.120:0/3081679864' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-06-23T01:11:03.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:02 smithi120 ceph-mon[79813]: from='mgr.14100 172.21.15.120:0/3081679864' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-06-23T01:11:03.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:02 smithi120 ceph-mon[79813]: from='mgr.14100 172.21.15.120:0/3081679864' entity='mgr.x' 2023-06-23T01:11:03.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:02 smithi120 ceph-mon[79813]: from='mgr.14100 172.21.15.120:0/3081679864' entity='mgr.x' 2023-06-23T01:11:03.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:02 smithi120 ceph-mon[79813]: from='mgr.14100 172.21.15.120:0/3081679864' entity='mgr.x' 2023-06-23T01:11:04.622 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 2023-06-23T01:11:04.622 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout { 2023-06-23T01:11:04.622 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "fsid": "9efbfa8e-1162-11ee-9b2b-001a4aab830c", 2023-06-23T01:11:04.623 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "health": { 2023-06-23T01:11:04.623 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-06-23T01:11:04.623 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-06-23T01:11:04.623 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-06-23T01:11:04.623 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2023-06-23T01:11:04.623 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-06-23T01:11:04.623 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-06-23T01:11:04.624 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 0 2023-06-23T01:11:04.624 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout ], 2023-06-23T01:11:04.624 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-06-23T01:11:04.624 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "a" 2023-06-23T01:11:04.624 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout ], 2023-06-23T01:11:04.624 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2023-06-23T01:11:04.624 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "monmap": { 2023-06-23T01:11:04.624 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-23T01:11:04.625 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2023-06-23T01:11:04.625 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-06-23T01:11:04.625 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2023-06-23T01:11:04.625 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-06-23T01:11:04.625 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-23T01:11:04.625 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-06-23T01:11:04.625 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-06-23T01:11:04.626 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-06-23T01:11:04.632 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-06-23T01:11:04.632 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-06-23T01:11:04.633 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-06-23T01:11:04.633 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2023-06-23T01:11:04.633 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-06-23T01:11:04.633 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-06-23T01:11:04.633 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-06-23T01:11:04.633 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-06-23T01:11:04.633 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-06-23T01:11:04.633 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-06-23T01:11:04.634 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-06-23T01:11:04.634 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-06-23T01:11:04.634 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-06-23T01:11:04.634 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2023-06-23T01:11:04.634 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-06-23T01:11:04.634 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-23T01:11:04.634 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-06-23T01:11:04.635 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-06-23T01:11:04.635 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2023-06-23T01:11:04.635 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-06-23T01:11:04.635 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "available": true, 2023-06-23T01:11:04.635 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-06-23T01:11:04.635 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "modules": [ 2023-06-23T01:11:04.635 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "iostat", 2023-06-23T01:11:04.635 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "nfs", 2023-06-23T01:11:04.636 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "restful" 2023-06-23T01:11:04.637 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout ], 2023-06-23T01:11:04.637 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "services": {} 2023-06-23T01:11:04.637 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2023-06-23T01:11:04.637 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-06-23T01:11:04.637 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-23T01:11:04.638 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "modified": "2023-06-23T01:10:46.841001+0000", 2023-06-23T01:11:04.638 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "services": {} 2023-06-23T01:11:04.638 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2023-06-23T01:11:04.638 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-06-23T01:11:04.638 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout } 2023-06-23T01:11:04.638 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.955+0000 7fb8fffff700 1 Processor -- start 2023-06-23T01:11:04.638 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.956+0000 7fb8fffff700 1 -- start start 2023-06-23T01:11:04.639 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.956+0000 7fb8fffff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb900070e10 0x7fb9001070e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:04.639 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.956+0000 7fb8fffff700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb900102ad0 con 0x7fb900070e10 2023-06-23T01:11:04.639 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.956+0000 7fb8feffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb900070e10 0x7fb9001070e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:04.639 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.956+0000 7fb8feffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb900070e10 0x7fb9001070e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49388/0 (socket says 172.21.15.120:49388) 2023-06-23T01:11:04.639 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.956+0000 7fb8feffd700 1 -- 172.21.15.120:0/777531491 learned_addr learned my addr 172.21.15.120:0/777531491 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:04.639 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.957+0000 7fb8feffd700 1 -- 172.21.15.120:0/777531491 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb900107620 con 0x7fb900070e10 2023-06-23T01:11:04.639 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.957+0000 7fb8feffd700 1 --2- 172.21.15.120:0/777531491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb900070e10 0x7fb9001070e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0009e50 tx=0x7fb8f0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e86f3ed21d5ce97 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:04.640 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.957+0000 7fb8fdffb700 1 -- 172.21.15.120:0/777531491 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb8f000e070 con 0x7fb900070e10 2023-06-23T01:11:04.640 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.957+0000 7fb8fdffb700 1 -- 172.21.15.120:0/777531491 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb8f0012430 con 0x7fb900070e10 2023-06-23T01:11:04.640 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.957+0000 7fb8fdffb700 1 -- 172.21.15.120:0/777531491 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb8f0007400 con 0x7fb900070e10 2023-06-23T01:11:04.640 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.958+0000 7fb8fffff700 1 -- 172.21.15.120:0/777531491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb900070e10 msgr2=0x7fb9001070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:04.640 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.958+0000 7fb8fffff700 1 --2- 172.21.15.120:0/777531491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb900070e10 0x7fb9001070e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0009e50 tx=0x7fb8f0009840 comp rx=0 tx=0).stop 2023-06-23T01:11:04.640 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.958+0000 7fb8fffff700 1 -- 172.21.15.120:0/777531491 shutdown_connections 2023-06-23T01:11:04.640 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.958+0000 7fb8fffff700 1 --2- 172.21.15.120:0/777531491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb900070e10 0x7fb9001070e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:04.640 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.958+0000 7fb8fffff700 1 -- 172.21.15.120:0/777531491 >> 172.21.15.120:0/777531491 conn(0x7fb9000fc6f0 msgr2=0x7fb9000feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:04.641 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.958+0000 7fb8fffff700 1 -- 172.21.15.120:0/777531491 shutdown_connections 2023-06-23T01:11:04.641 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.958+0000 7fb8fffff700 1 -- 172.21.15.120:0/777531491 wait complete. 2023-06-23T01:11:04.641 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.959+0000 7fb8fffff700 1 Processor -- start 2023-06-23T01:11:04.641 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.959+0000 7fb8fffff700 1 -- start start 2023-06-23T01:11:04.641 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.959+0000 7fb8fffff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb900070e10 0x7fb900112fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:04.641 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.959+0000 7fb8fffff700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb900113500 con 0x7fb900070e10 2023-06-23T01:11:04.641 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.959+0000 7fb8feffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb900070e10 0x7fb900112fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:04.642 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.960+0000 7fb8feffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb900070e10 0x7fb900112fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49390/0 (socket says 172.21.15.120:49390) 2023-06-23T01:11:04.642 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.960+0000 7fb8feffd700 1 -- 172.21.15.120:0/3155783817 learned_addr learned my addr 172.21.15.120:0/3155783817 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:04.642 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.960+0000 7fb8feffd700 1 -- 172.21.15.120:0/3155783817 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f0009ae0 con 0x7fb900070e10 2023-06-23T01:11:04.642 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.960+0000 7fb8feffd700 1 --2- 172.21.15.120:0/3155783817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb900070e10 0x7fb900112fc0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0009fd0 tx=0x7fb8f0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:04.642 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.960+0000 7fb8e77fe700 1 -- 172.21.15.120:0/3155783817 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb8f000e030 con 0x7fb900070e10 2023-06-23T01:11:04.642 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.960+0000 7fb8e77fe700 1 -- 172.21.15.120:0/3155783817 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb8f000bc30 con 0x7fb900070e10 2023-06-23T01:11:04.642 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.960+0000 7fb8fffff700 1 -- 172.21.15.120:0/3155783817 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb900113700 con 0x7fb900070e10 2023-06-23T01:11:04.642 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.960+0000 7fb8e77fe700 1 -- 172.21.15.120:0/3155783817 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb8f001c530 con 0x7fb900070e10 2023-06-23T01:11:04.643 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.960+0000 7fb8fffff700 1 -- 172.21.15.120:0/3155783817 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9001143a0 con 0x7fb900070e10 2023-06-23T01:11:04.643 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.961+0000 7fb8fffff700 1 -- 172.21.15.120:0/3155783817 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb90005f060 con 0x7fb900070e10 2023-06-23T01:11:04.643 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.961+0000 7fb8e77fe700 1 -- 172.21.15.120:0/3155783817 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 3) v1 ==== 43102+0+0 (secure 0 0 0) 0x7fb8f0019070 con 0x7fb900070e10 2023-06-23T01:11:04.643 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.962+0000 7fb8e77fe700 1 --2- 172.21.15.120:0/3155783817 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7fb8e8036450 0x7fb8e8038900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:04.643 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.962+0000 7fb8e77fe700 1 -- 172.21.15.120:0/3155783817 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fb8f00495e0 con 0x7fb900070e10 2023-06-23T01:11:04.643 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.962+0000 7fb8fe7fc700 1 --2- 172.21.15.120:0/3155783817 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7fb8e8036450 0x7fb8e8038900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:04.643 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.963+0000 7fb8fe7fc700 1 --2- 172.21.15.120:0/3155783817 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7fb8e8036450 0x7fb8e8038900 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4009990 tx=0x7fb8f4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:04.644 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:03.968+0000 7fb8e77fe700 1 -- 172.21.15.120:0/3155783817 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fb8f0019930 con 0x7fb900070e10 2023-06-23T01:11:04.644 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:04.233+0000 7fb8fffff700 1 -- 172.21.15.120:0/3155783817 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb900114740 con 0x7fb900070e10 2023-06-23T01:11:04.644 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:04.234+0000 7fb8e77fe700 1 -- 172.21.15.120:0/3155783817 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7fb8f00204f0 con 0x7fb900070e10 2023-06-23T01:11:04.644 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:04.236+0000 7fb8fffff700 1 -- 172.21.15.120:0/3155783817 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7fb8e8036450 msgr2=0x7fb8e8038900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:04.644 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:04.236+0000 7fb8fffff700 1 --2- 172.21.15.120:0/3155783817 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7fb8e8036450 0x7fb8e8038900 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4009990 tx=0x7fb8f4006e40 comp rx=0 tx=0).stop 2023-06-23T01:11:04.644 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:04.236+0000 7fb8fffff700 1 -- 172.21.15.120:0/3155783817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb900070e10 msgr2=0x7fb900112fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:04.644 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:04.236+0000 7fb8fffff700 1 --2- 172.21.15.120:0/3155783817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb900070e10 0x7fb900112fc0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0009fd0 tx=0x7fb8f0009840 comp rx=0 tx=0).stop 2023-06-23T01:11:04.645 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:04.236+0000 7fb8fffff700 1 -- 172.21.15.120:0/3155783817 shutdown_connections 2023-06-23T01:11:04.645 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:04.236+0000 7fb8fffff700 1 --2- 172.21.15.120:0/3155783817 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7fb8e8036450 0x7fb8e8038900 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:04.645 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:04.236+0000 7fb8fffff700 1 --2- 172.21.15.120:0/3155783817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb900070e10 0x7fb900112fc0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:04.645 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:04.236+0000 7fb8fffff700 1 -- 172.21.15.120:0/3155783817 >> 172.21.15.120:0/3155783817 conn(0x7fb9000fc6f0 msgr2=0x7fb9000fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:04.645 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:04.237+0000 7fb8fffff700 1 -- 172.21.15.120:0/3155783817 shutdown_connections 2023-06-23T01:11:04.645 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:04.237+0000 7fb8fffff700 1 -- 172.21.15.120:0/3155783817 wait complete. 2023-06-23T01:11:04.645 INFO:teuthology.orchestra.run.smithi120.stdout:mgr is available 2023-06-23T01:11:04.659 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:04 smithi120 ceph-mon[79813]: mgrmap e3: x(active, since 1.00893s) 2023-06-23T01:11:04.659 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:04 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/3155783817' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-06-23T01:11:05.703 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 2023-06-23T01:11:05.704 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout [global] 2023-06-23T01:11:05.704 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout fsid = 9efbfa8e-1162-11ee-9b2b-001a4aab830c 2023-06-23T01:11:05.704 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-06-23T01:11:05.704 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-06-23T01:11:05.704 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-06-23T01:11:05.704 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-06-23T01:11:05.705 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 2023-06-23T01:11:05.705 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout [mgr] 2023-06-23T01:11:05.705 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-06-23T01:11:05.705 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 2023-06-23T01:11:05.705 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout [osd] 2023-06-23T01:11:05.705 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-06-23T01:11:05.705 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-06-23T01:11:05.705 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-06-23T01:11:05.706 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-06-23T01:11:05.706 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-06-23T01:11:05.706 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.140+0000 7fcd7880d700 1 Processor -- start 2023-06-23T01:11:05.706 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.141+0000 7fcd7880d700 1 -- start start 2023-06-23T01:11:05.706 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.141+0000 7fcd7880d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd74077cf0 0x7fcd74076190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:05.706 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.141+0000 7fcd7880d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd740766d0 con 0x7fcd74077cf0 2023-06-23T01:11:05.706 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.141+0000 7fcd7259c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd74077cf0 0x7fcd74076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:05.707 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.141+0000 7fcd7259c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd74077cf0 0x7fcd74076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49398/0 (socket says 172.21.15.120:49398) 2023-06-23T01:11:05.707 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.141+0000 7fcd7259c700 1 -- 172.21.15.120:0/2168215918 learned_addr learned my addr 172.21.15.120:0/2168215918 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:05.707 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.142+0000 7fcd7259c700 1 -- 172.21.15.120:0/2168215918 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd74076810 con 0x7fcd74077cf0 2023-06-23T01:11:05.707 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.142+0000 7fcd7259c700 1 --2- 172.21.15.120:0/2168215918 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd74077cf0 0x7fcd74076190 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcd640098e0 tx=0x7fcd64009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=543fb381f346c698 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:05.707 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.142+0000 7fcd71d9b700 1 -- 172.21.15.120:0/2168215918 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcd64009e80 con 0x7fcd74077cf0 2023-06-23T01:11:05.707 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.142+0000 7fcd71d9b700 1 -- 172.21.15.120:0/2168215918 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcd6400bd10 con 0x7fcd74077cf0 2023-06-23T01:11:05.707 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.142+0000 7fcd71d9b700 1 -- 172.21.15.120:0/2168215918 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcd64011ce0 con 0x7fcd74077cf0 2023-06-23T01:11:05.707 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.142+0000 7fcd7880d700 1 -- 172.21.15.120:0/2168215918 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd74077cf0 msgr2=0x7fcd74076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:05.708 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.142+0000 7fcd7880d700 1 --2- 172.21.15.120:0/2168215918 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd74077cf0 0x7fcd74076190 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcd640098e0 tx=0x7fcd64009250 comp rx=0 tx=0).stop 2023-06-23T01:11:05.708 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.143+0000 7fcd7880d700 1 -- 172.21.15.120:0/2168215918 shutdown_connections 2023-06-23T01:11:05.708 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.143+0000 7fcd7880d700 1 --2- 172.21.15.120:0/2168215918 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd74077cf0 0x7fcd74076190 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:05.708 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.143+0000 7fcd7880d700 1 -- 172.21.15.120:0/2168215918 >> 172.21.15.120:0/2168215918 conn(0x7fcd740fd990 msgr2=0x7fcd740ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:05.708 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.143+0000 7fcd7880d700 1 -- 172.21.15.120:0/2168215918 shutdown_connections 2023-06-23T01:11:05.708 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.143+0000 7fcd7880d700 1 -- 172.21.15.120:0/2168215918 wait complete. 2023-06-23T01:11:05.708 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.143+0000 7fcd7880d700 1 Processor -- start 2023-06-23T01:11:05.709 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.144+0000 7fcd7880d700 1 -- start start 2023-06-23T01:11:05.709 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.144+0000 7fcd7880d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd74077cf0 0x7fcd74113120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:05.709 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.144+0000 7fcd7880d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd74113660 con 0x7fcd74077cf0 2023-06-23T01:11:05.709 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.144+0000 7fcd7259c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd74077cf0 0x7fcd74113120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:05.709 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.144+0000 7fcd7259c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd74077cf0 0x7fcd74113120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49406/0 (socket says 172.21.15.120:49406) 2023-06-23T01:11:05.709 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.144+0000 7fcd7259c700 1 -- 172.21.15.120:0/2189472298 learned_addr learned my addr 172.21.15.120:0/2189472298 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:05.709 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.144+0000 7fcd7259c700 1 -- 172.21.15.120:0/2189472298 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd640094f0 con 0x7fcd74077cf0 2023-06-23T01:11:05.709 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.145+0000 7fcd7259c700 1 --2- 172.21.15.120:0/2189472298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd74077cf0 0x7fcd74113120 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcd64011970 tx=0x7fcd64009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:05.710 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.145+0000 7fcd6affd700 1 -- 172.21.15.120:0/2189472298 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcd64009e80 con 0x7fcd74077cf0 2023-06-23T01:11:05.710 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.145+0000 7fcd6affd700 1 -- 172.21.15.120:0/2189472298 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcd6401aaf0 con 0x7fcd74077cf0 2023-06-23T01:11:05.710 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.145+0000 7fcd7880d700 1 -- 172.21.15.120:0/2189472298 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd74113860 con 0x7fcd74077cf0 2023-06-23T01:11:05.710 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.145+0000 7fcd6affd700 1 -- 172.21.15.120:0/2189472298 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcd64022a90 con 0x7fcd74077cf0 2023-06-23T01:11:05.710 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.145+0000 7fcd7880d700 1 -- 172.21.15.120:0/2189472298 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd74114500 con 0x7fcd74077cf0 2023-06-23T01:11:05.710 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.146+0000 7fcd6affd700 1 -- 172.21.15.120:0/2189472298 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 4) v1 ==== 43208+0+0 (secure 0 0 0) 0x7fcd64022c90 con 0x7fcd74077cf0 2023-06-23T01:11:05.710 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.146+0000 7fcd7880d700 1 -- 172.21.15.120:0/2189472298 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd7405f060 con 0x7fcd74077cf0 2023-06-23T01:11:05.711 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.147+0000 7fcd6affd700 1 --2- 172.21.15.120:0/2189472298 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7fcd60036520 0x7fcd600389d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:05.711 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.147+0000 7fcd6affd700 1 -- 172.21.15.120:0/2189472298 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fcd6404a340 con 0x7fcd74077cf0 2023-06-23T01:11:05.711 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.147+0000 7fcd6bfff700 1 --2- 172.21.15.120:0/2189472298 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7fcd60036520 0x7fcd600389d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:05.711 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.148+0000 7fcd6bfff700 1 --2- 172.21.15.120:0/2189472298 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7fcd60036520 0x7fcd600389d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c009990 tx=0x7fcd5c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:05.711 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.151+0000 7fcd6affd700 1 -- 172.21.15.120:0/2189472298 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fcd6401ac60 con 0x7fcd74077cf0 2023-06-23T01:11:05.711 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.341+0000 7fcd7880d700 1 -- 172.21.15.120:0/2189472298 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fcd7404b970 con 0x7fcd74077cf0 2023-06-23T01:11:05.711 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.341+0000 7fcd6affd700 1 -- 172.21.15.120:0/2189472298 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fcd640213a0 con 0x7fcd74077cf0 2023-06-23T01:11:05.711 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.344+0000 7fcd7880d700 1 -- 172.21.15.120:0/2189472298 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7fcd60036520 msgr2=0x7fcd600389d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:05.712 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.344+0000 7fcd7880d700 1 --2- 172.21.15.120:0/2189472298 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7fcd60036520 0x7fcd600389d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c009990 tx=0x7fcd5c006e40 comp rx=0 tx=0).stop 2023-06-23T01:11:05.712 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.344+0000 7fcd7880d700 1 -- 172.21.15.120:0/2189472298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd74077cf0 msgr2=0x7fcd74113120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:05.712 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.344+0000 7fcd7880d700 1 --2- 172.21.15.120:0/2189472298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd74077cf0 0x7fcd74113120 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcd64011970 tx=0x7fcd64009250 comp rx=0 tx=0).stop 2023-06-23T01:11:05.712 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.344+0000 7fcd7880d700 1 -- 172.21.15.120:0/2189472298 shutdown_connections 2023-06-23T01:11:05.712 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.344+0000 7fcd7880d700 1 --2- 172.21.15.120:0/2189472298 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7fcd60036520 0x7fcd600389d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:05.712 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.344+0000 7fcd7880d700 1 --2- 172.21.15.120:0/2189472298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd74077cf0 0x7fcd74113120 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:05.713 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.344+0000 7fcd7880d700 1 -- 172.21.15.120:0/2189472298 >> 172.21.15.120:0/2189472298 conn(0x7fcd740fd990 msgr2=0x7fcd740fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:05.713 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.344+0000 7fcd7880d700 1 -- 172.21.15.120:0/2189472298 shutdown_connections 2023-06-23T01:11:05.713 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:05.344+0000 7fcd7880d700 1 -- 172.21.15.120:0/2189472298 wait complete. 2023-06-23T01:11:05.713 INFO:teuthology.orchestra.run.smithi120.stdout:Enabling cephadm module... 2023-06-23T01:11:05.760 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:05 smithi120 ceph-mon[79813]: mgrmap e4: x(active, since 2s) 2023-06-23T01:11:05.760 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:05 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/2189472298' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-06-23T01:11:06.760 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:06 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/1576222596' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-06-23T01:11:06.761 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:06 smithi120 conmon[80224]: ignoring --setuser ceph since I am not root 2023-06-23T01:11:06.761 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:06 smithi120 conmon[80224]: ignoring --setgroup ceph since I am not root 2023-06-23T01:11:06.873 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.219+0000 7f81465f7700 1 Processor -- start 2023-06-23T01:11:06.873 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.219+0000 7f81465f7700 1 -- start start 2023-06-23T01:11:06.874 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.220+0000 7f81465f7700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8140102270 0x7f8140102680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:06.874 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.220+0000 7f81465f7700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8140102bc0 con 0x7f8140102270 2023-06-23T01:11:06.874 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.220+0000 7f813ffff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8140102270 0x7f8140102680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:06.874 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.220+0000 7f813ffff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8140102270 0x7f8140102680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49416/0 (socket says 172.21.15.120:49416) 2023-06-23T01:11:06.874 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.220+0000 7f813ffff700 1 -- 172.21.15.120:0/1508103093 learned_addr learned my addr 172.21.15.120:0/1508103093 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:06.874 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.221+0000 7f813ffff700 1 -- 172.21.15.120:0/1508103093 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8140102d00 con 0x7f8140102270 2023-06-23T01:11:06.875 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.221+0000 7f813ffff700 1 --2- 172.21.15.120:0/1508103093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8140102270 0x7f8140102680 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f81300098e0 tx=0x7f8130009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=544e6b33ff87f61b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:06.875 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.221+0000 7f813effd700 1 -- 172.21.15.120:0/1508103093 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8130009e80 con 0x7f8140102270 2023-06-23T01:11:06.875 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.221+0000 7f813effd700 1 -- 172.21.15.120:0/1508103093 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f813000bd10 con 0x7f8140102270 2023-06-23T01:11:06.875 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.221+0000 7f813effd700 1 -- 172.21.15.120:0/1508103093 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8130011ce0 con 0x7f8140102270 2023-06-23T01:11:06.875 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.222+0000 7f81465f7700 1 -- 172.21.15.120:0/1508103093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8140102270 msgr2=0x7f8140102680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:06.875 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.222+0000 7f81465f7700 1 --2- 172.21.15.120:0/1508103093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8140102270 0x7f8140102680 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f81300098e0 tx=0x7f8130009250 comp rx=0 tx=0).stop 2023-06-23T01:11:06.875 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.222+0000 7f81465f7700 1 -- 172.21.15.120:0/1508103093 shutdown_connections 2023-06-23T01:11:06.875 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.222+0000 7f81465f7700 1 --2- 172.21.15.120:0/1508103093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8140102270 0x7f8140102680 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:06.876 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.222+0000 7f81465f7700 1 -- 172.21.15.120:0/1508103093 >> 172.21.15.120:0/1508103093 conn(0x7f81400fd9f0 msgr2=0x7f81400ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:06.876 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.222+0000 7f81465f7700 1 -- 172.21.15.120:0/1508103093 shutdown_connections 2023-06-23T01:11:06.876 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.222+0000 7f81465f7700 1 -- 172.21.15.120:0/1508103093 wait complete. 2023-06-23T01:11:06.876 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.222+0000 7f81465f7700 1 Processor -- start 2023-06-23T01:11:06.876 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.223+0000 7f81465f7700 1 -- start start 2023-06-23T01:11:06.876 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.223+0000 7f81465f7700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8140102270 0x7f8140110e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:06.876 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.223+0000 7f81465f7700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81401113d0 con 0x7f8140102270 2023-06-23T01:11:06.877 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.223+0000 7f813ffff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8140102270 0x7f8140110e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:06.877 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.223+0000 7f813ffff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8140102270 0x7f8140110e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49424/0 (socket says 172.21.15.120:49424) 2023-06-23T01:11:06.877 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.223+0000 7f813ffff700 1 -- 172.21.15.120:0/1576222596 learned_addr learned my addr 172.21.15.120:0/1576222596 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:06.877 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.223+0000 7f813ffff700 1 -- 172.21.15.120:0/1576222596 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81300094f0 con 0x7f8140102270 2023-06-23T01:11:06.877 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.224+0000 7f813ffff700 1 --2- 172.21.15.120:0/1576222596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8140102270 0x7f8140110e90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8130006b00 tx=0x7f8130009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:06.877 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.224+0000 7f813cff9700 1 -- 172.21.15.120:0/1576222596 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f813001a640 con 0x7f8140102270 2023-06-23T01:11:06.877 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.224+0000 7f81465f7700 1 -- 172.21.15.120:0/1576222596 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81401115d0 con 0x7f8140102270 2023-06-23T01:11:06.877 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.224+0000 7f813cff9700 1 -- 172.21.15.120:0/1576222596 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f813001ac50 con 0x7f8140102270 2023-06-23T01:11:06.878 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.224+0000 7f813cff9700 1 -- 172.21.15.120:0/1576222596 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8130022ab0 con 0x7f8140102270 2023-06-23T01:11:06.878 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.224+0000 7f81465f7700 1 -- 172.21.15.120:0/1576222596 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8140112270 con 0x7f8140102270 2023-06-23T01:11:06.878 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.225+0000 7f813cff9700 1 -- 172.21.15.120:0/1576222596 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 4) v1 ==== 43208+0+0 (secure 0 0 0) 0x7f8130022d20 con 0x7f8140102270 2023-06-23T01:11:06.878 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.225+0000 7f81465f7700 1 -- 172.21.15.120:0/1576222596 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f814005f060 con 0x7f8140102270 2023-06-23T01:11:06.878 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.225+0000 7f813cff9700 1 --2- 172.21.15.120:0/1576222596 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7f8128036560 0x7f8128038a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:06.878 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.226+0000 7f813cff9700 1 -- 172.21.15.120:0/1576222596 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f813004a610 con 0x7f8140102270 2023-06-23T01:11:06.878 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.226+0000 7f813f7fe700 1 --2- 172.21.15.120:0/1576222596 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7f8128036560 0x7f8128038a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:06.879 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.226+0000 7f813f7fe700 1 --2- 172.21.15.120:0/1576222596 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7f8128036560 0x7f8128038a10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8134009990 tx=0x7f8134006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:06.879 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.230+0000 7f813cff9700 1 -- 172.21.15.120:0/1576222596 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f813002c440 con 0x7f8140102270 2023-06-23T01:11:06.879 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.471+0000 7f81465f7700 1 -- 172.21.15.120:0/1576222596 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f814004b970 con 0x7f8140102270 2023-06-23T01:11:06.879 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.480+0000 7f813cff9700 1 -- 172.21.15.120:0/1576222596 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mgrmap(e 5) v1 ==== 43219+0+0 (secure 0 0 0) 0x7f8130022400 con 0x7f8140102270 2023-06-23T01:11:06.879 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.481+0000 7f813cff9700 1 -- 172.21.15.120:0/1576222596 <== mon.0 v2:172.21.15.120:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f813004c440 con 0x7f8140102270 2023-06-23T01:11:06.879 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.484+0000 7f81465f7700 1 -- 172.21.15.120:0/1576222596 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7f8128036560 msgr2=0x7f8128038a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:06.879 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.484+0000 7f81465f7700 1 --2- 172.21.15.120:0/1576222596 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7f8128036560 0x7f8128038a10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8134009990 tx=0x7f8134006e40 comp rx=0 tx=0).stop 2023-06-23T01:11:06.879 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.484+0000 7f81465f7700 1 -- 172.21.15.120:0/1576222596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8140102270 msgr2=0x7f8140110e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:06.880 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.484+0000 7f81465f7700 1 --2- 172.21.15.120:0/1576222596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8140102270 0x7f8140110e90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8130006b00 tx=0x7f8130009250 comp rx=0 tx=0).stop 2023-06-23T01:11:06.880 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.484+0000 7f81465f7700 1 -- 172.21.15.120:0/1576222596 shutdown_connections 2023-06-23T01:11:06.880 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.484+0000 7f81465f7700 1 --2- 172.21.15.120:0/1576222596 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7f8128036560 0x7f8128038a10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:06.880 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.484+0000 7f81465f7700 1 --2- 172.21.15.120:0/1576222596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8140102270 0x7f8140110e90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:06.880 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.484+0000 7f81465f7700 1 -- 172.21.15.120:0/1576222596 >> 172.21.15.120:0/1576222596 conn(0x7f81400fd9f0 msgr2=0x7f81400fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:06.880 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.484+0000 7f81465f7700 1 -- 172.21.15.120:0/1576222596 shutdown_connections 2023-06-23T01:11:06.880 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:06.484+0000 7f81465f7700 1 -- 172.21.15.120:0/1576222596 wait complete. 2023-06-23T01:11:07.760 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:07 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/1576222596' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-06-23T01:11:07.760 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:07 smithi120 ceph-mon[79813]: mgrmap e5: x(active, since 4s) 2023-06-23T01:11:08.047 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout { 2023-06-23T01:11:08.048 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-06-23T01:11:08.048 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "available": true, 2023-06-23T01:11:08.048 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-06-23T01:11:08.048 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-06-23T01:11:08.048 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout } 2023-06-23T01:11:08.048 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.377+0000 7f787268c700 1 Processor -- start 2023-06-23T01:11:08.049 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.377+0000 7f787268c700 1 -- start start 2023-06-23T01:11:08.049 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.378+0000 7f787268c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f786c077cd0 0x7f786c076170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:08.049 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.378+0000 7f787268c700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c0766b0 con 0x7f786c077cd0 2023-06-23T01:11:08.049 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.378+0000 7f786bfff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f786c077cd0 0x7f786c076170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:08.049 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.378+0000 7f786bfff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f786c077cd0 0x7f786c076170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49456/0 (socket says 172.21.15.120:49456) 2023-06-23T01:11:08.049 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.378+0000 7f786bfff700 1 -- 172.21.15.120:0/3593967321 learned_addr learned my addr 172.21.15.120:0/3593967321 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:08.049 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.379+0000 7f786bfff700 1 -- 172.21.15.120:0/3593967321 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786c0767f0 con 0x7f786c077cd0 2023-06-23T01:11:08.049 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.379+0000 7f786bfff700 1 --2- 172.21.15.120:0/3593967321 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f786c077cd0 0x7f786c076170 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f785c0098e0 tx=0x7f785c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10622480f50ce2bf server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:08.050 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.379+0000 7f786affd700 1 -- 172.21.15.120:0/3593967321 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f785c009e80 con 0x7f786c077cd0 2023-06-23T01:11:08.050 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.379+0000 7f786affd700 1 -- 172.21.15.120:0/3593967321 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f785c00bd10 con 0x7f786c077cd0 2023-06-23T01:11:08.050 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.379+0000 7f786affd700 1 -- 172.21.15.120:0/3593967321 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f785c011ce0 con 0x7f786c077cd0 2023-06-23T01:11:08.050 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.380+0000 7f787268c700 1 -- 172.21.15.120:0/3593967321 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f786c077cd0 msgr2=0x7f786c076170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:08.050 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.380+0000 7f787268c700 1 --2- 172.21.15.120:0/3593967321 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f786c077cd0 0x7f786c076170 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f785c0098e0 tx=0x7f785c009250 comp rx=0 tx=0).stop 2023-06-23T01:11:08.050 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.380+0000 7f787268c700 1 -- 172.21.15.120:0/3593967321 shutdown_connections 2023-06-23T01:11:08.050 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.380+0000 7f787268c700 1 --2- 172.21.15.120:0/3593967321 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f786c077cd0 0x7f786c076170 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:08.051 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.380+0000 7f787268c700 1 -- 172.21.15.120:0/3593967321 >> 172.21.15.120:0/3593967321 conn(0x7f786c0fd990 msgr2=0x7f786c0ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:08.051 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.380+0000 7f787268c700 1 -- 172.21.15.120:0/3593967321 shutdown_connections 2023-06-23T01:11:08.051 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.380+0000 7f787268c700 1 -- 172.21.15.120:0/3593967321 wait complete. 2023-06-23T01:11:08.051 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.381+0000 7f787268c700 1 Processor -- start 2023-06-23T01:11:08.051 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.381+0000 7f787268c700 1 -- start start 2023-06-23T01:11:08.051 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.381+0000 7f787268c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f786c077cd0 0x7f786c113100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:08.051 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.381+0000 7f787268c700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c113640 con 0x7f786c077cd0 2023-06-23T01:11:08.051 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.381+0000 7f786bfff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f786c077cd0 0x7f786c113100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:08.052 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.381+0000 7f786bfff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f786c077cd0 0x7f786c113100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49466/0 (socket says 172.21.15.120:49466) 2023-06-23T01:11:08.052 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.381+0000 7f786bfff700 1 -- 172.21.15.120:0/2070498293 learned_addr learned my addr 172.21.15.120:0/2070498293 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:08.052 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.382+0000 7f786bfff700 1 -- 172.21.15.120:0/2070498293 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f785c0094f0 con 0x7f786c077cd0 2023-06-23T01:11:08.052 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.382+0000 7f786bfff700 1 --2- 172.21.15.120:0/2070498293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f786c077cd0 0x7f786c113100 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f785c011970 tx=0x7f785c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:08.052 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.382+0000 7f7868ff9700 1 -- 172.21.15.120:0/2070498293 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f785c009e80 con 0x7f786c077cd0 2023-06-23T01:11:08.052 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.382+0000 7f7868ff9700 1 -- 172.21.15.120:0/2070498293 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f785c01aaf0 con 0x7f786c077cd0 2023-06-23T01:11:08.052 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.382+0000 7f787268c700 1 -- 172.21.15.120:0/2070498293 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f786c113840 con 0x7f786c077cd0 2023-06-23T01:11:08.053 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.382+0000 7f7868ff9700 1 -- 172.21.15.120:0/2070498293 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f785c022a90 con 0x7f786c077cd0 2023-06-23T01:11:08.053 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.382+0000 7f787268c700 1 -- 172.21.15.120:0/2070498293 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f786c1144e0 con 0x7f786c077cd0 2023-06-23T01:11:08.055 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.383+0000 7f7868ff9700 1 -- 172.21.15.120:0/2070498293 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) v1 ==== 43219+0+0 (secure 0 0 0) 0x7f785c022c90 con 0x7f786c077cd0 2023-06-23T01:11:08.055 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.383+0000 7f787268c700 1 -- 172.21.15.120:0/2070498293 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f786c05f060 con 0x7f786c077cd0 2023-06-23T01:11:08.055 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.384+0000 7f7868ff9700 1 --2- 172.21.15.120:0/2070498293 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7f7854036560 0x7f7854038a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:08.055 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.384+0000 7f7868ff9700 1 -- 172.21.15.120:0/2070498293 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f785c04a390 con 0x7f786c077cd0 2023-06-23T01:11:08.055 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.384+0000 7f786b7fe700 1 -- 172.21.15.120:0/2070498293 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7f7854036560 msgr2=0x7f7854038a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.120:6800/506843621 2023-06-23T01:11:08.055 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.384+0000 7f786b7fe700 1 --2- 172.21.15.120:0/2070498293 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7f7854036560 0x7f7854038a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-23T01:11:08.056 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.389+0000 7f7868ff9700 1 -- 172.21.15.120:0/2070498293 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f785c006d70 con 0x7f786c077cd0 2023-06-23T01:11:08.056 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.585+0000 7f786b7fe700 1 -- 172.21.15.120:0/2070498293 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7f7854036560 msgr2=0x7f7854038a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.120:6800/506843621 2023-06-23T01:11:08.056 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.585+0000 7f786b7fe700 1 --2- 172.21.15.120:0/2070498293 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7f7854036560 0x7f7854038a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-06-23T01:11:08.056 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.634+0000 7f787268c700 1 -- 172.21.15.120:0/2070498293 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f786c114820 con 0x7f786c077cd0 2023-06-23T01:11:08.056 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.634+0000 7f7868ff9700 1 -- 172.21.15.120:0/2070498293 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f785c0213a0 con 0x7f786c077cd0 2023-06-23T01:11:08.056 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.636+0000 7f787268c700 1 -- 172.21.15.120:0/2070498293 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7f7854036560 msgr2=0x7f7854038a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-23T01:11:08.056 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.637+0000 7f787268c700 1 --2- 172.21.15.120:0/2070498293 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7f7854036560 0x7f7854038a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:08.056 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.637+0000 7f787268c700 1 -- 172.21.15.120:0/2070498293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f786c077cd0 msgr2=0x7f786c113100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:08.057 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.637+0000 7f787268c700 1 --2- 172.21.15.120:0/2070498293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f786c077cd0 0x7f786c113100 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f785c011970 tx=0x7f785c009250 comp rx=0 tx=0).stop 2023-06-23T01:11:08.057 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.637+0000 7f787268c700 1 -- 172.21.15.120:0/2070498293 shutdown_connections 2023-06-23T01:11:08.057 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.637+0000 7f787268c700 1 --2- 172.21.15.120:0/2070498293 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7f7854036560 0x7f7854038a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:08.057 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.637+0000 7f787268c700 1 --2- 172.21.15.120:0/2070498293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f786c077cd0 0x7f786c113100 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:08.057 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.637+0000 7f787268c700 1 -- 172.21.15.120:0/2070498293 >> 172.21.15.120:0/2070498293 conn(0x7f786c0fd990 msgr2=0x7f786c0fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:08.057 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.637+0000 7f787268c700 1 -- 172.21.15.120:0/2070498293 shutdown_connections 2023-06-23T01:11:08.057 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:07.637+0000 7f787268c700 1 -- 172.21.15.120:0/2070498293 wait complete. 2023-06-23T01:11:08.057 INFO:teuthology.orchestra.run.smithi120.stdout:Waiting for the mgr to restart... 2023-06-23T01:11:08.058 INFO:teuthology.orchestra.run.smithi120.stdout:Waiting for mgr epoch 5... 2023-06-23T01:11:08.470 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:08 smithi120 conmon[80224]: 2023-06-23T01:11:08.125+0000 7fb7f2f701c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-06-23T01:11:08.470 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:08 smithi120 conmon[80224]: 2023-06-23T01:11:08.343+0000 7fb7f2f701c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-06-23T01:11:08.739 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:08 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/2070498293' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-06-23T01:11:08.739 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:08 smithi120 conmon[80224]: 2023-06-23T01:11:08.467+0000 7fb7f2f701c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-06-23T01:11:08.739 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:08 smithi120 conmon[80224]: 2023-06-23T01:11:08.735+0000 7fb7f2f701c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-06-23T01:11:09.010 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:08 smithi120 conmon[80224]: 2023-06-23T01:11:08.984+0000 7fb7f2f701c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-06-23T01:11:09.510 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:09 smithi120 conmon[80224]: 2023-06-23T01:11:09.175+0000 7fb7f2f701c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-06-23T01:11:09.510 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:09 smithi120 conmon[80224]: 2023-06-23T01:11:09.276+0000 7fb7f2f701c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-06-23T01:11:10.510 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:10 smithi120 conmon[80224]: 2023-06-23T01:11:10.015+0000 7fb7f2f701c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-06-23T01:11:11.010 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:10 smithi120 conmon[80224]: 2023-06-23T01:11:10.602+0000 7fb7f2f701c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-06-23T01:11:11.010 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:10 smithi120 conmon[80224]: 2023-06-23T01:11:10.717+0000 7fb7f2f701c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-06-23T01:11:11.010 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:10 smithi120 conmon[80224]: 2023-06-23T01:11:10.833+0000 7fb7f2f701c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-06-23T01:11:11.905 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:11 smithi120 conmon[80224]: 2023-06-23T01:11:11.576+0000 7fb7f2f701c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-06-23T01:11:11.905 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:11 smithi120 conmon[80224]: 2023-06-23T01:11:11.787+0000 7fb7f2f701c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-06-23T01:11:11.906 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:11 smithi120 conmon[80224]: 2023-06-23T01:11:11.902+0000 7fb7f2f701c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-06-23T01:11:12.184 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:12 smithi120 conmon[80224]: 2023-06-23T01:11:12.067+0000 7fb7f2f701c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-06-23T01:11:13.510 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:12 smithi120 conmon[80224]: 2023-06-23T01:11:12.181+0000 7fb7f2f701c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-06-23T01:11:13.510 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:13 smithi120 conmon[80224]: 2023-06-23T01:11:13.238+0000 7fb7f2f701c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-06-23T01:11:13.510 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:13 smithi120 conmon[80224]: 2023-06-23T01:11:13.351+0000 7fb7f2f701c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-06-23T01:11:13.511 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:13 smithi120 conmon[80224]: 2023-06-23T01:11:13.462+0000 7fb7f2f701c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-06-23T01:11:14.010 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:13 smithi120 conmon[80224]: 2023-06-23T01:11:13.647+0000 7fb7f2f701c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-06-23T01:11:14.010 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:13 smithi120 conmon[80224]: 2023-06-23T01:11:13.769+0000 7fb7f2f701c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-06-23T01:11:14.512 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:14 smithi120 conmon[80224]: 2023-06-23T01:11:14.234+0000 7fb7f2f701c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-06-23T01:11:14.513 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:14 smithi120 conmon[80224]: 2023-06-23T01:11:14.348+0000 7fb7f2f701c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-06-23T01:11:14.513 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:14 smithi120 conmon[80224]: 2023-06-23T01:11:14.460+0000 7fb7f2f701c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-06-23T01:11:15.010 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:14 smithi120 conmon[80224]: 2023-06-23T01:11:14.572+0000 7fb7f2f701c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-06-23T01:11:15.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:14 smithi120 ceph-mon[79813]: Active manager daemon x restarted 2023-06-23T01:11:15.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:14 smithi120 ceph-mon[79813]: Activating manager daemon x 2023-06-23T01:11:15.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:14 smithi120 ceph-mon[79813]: osdmap e2: 0 total, 0 up, 0 in 2023-06-23T01:11:15.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:14 smithi120 ceph-mon[79813]: mgrmap e6: x(active, starting, since 0.00374833s) 2023-06-23T01:11:15.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:14 smithi120 ceph-mon[79813]: from='mgr.14120 172.21.15.120:0/66681182' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-23T01:11:15.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:14 smithi120 ceph-mon[79813]: from='mgr.14120 172.21.15.120:0/66681182' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-06-23T01:11:15.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:14 smithi120 ceph-mon[79813]: from='mgr.14120 172.21.15.120:0/66681182' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-06-23T01:11:15.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:14 smithi120 ceph-mon[79813]: from='mgr.14120 172.21.15.120:0/66681182' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-06-23T01:11:15.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:14 smithi120 ceph-mon[79813]: from='mgr.14120 172.21.15.120:0/66681182' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-06-23T01:11:15.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:14 smithi120 ceph-mon[79813]: Manager daemon x is now available 2023-06-23T01:11:15.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:14 smithi120 ceph-mon[79813]: from='mgr.14120 172.21.15.120:0/66681182' entity='mgr.x' 2023-06-23T01:11:15.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:14 smithi120 ceph-mon[79813]: from='mgr.14120 172.21.15.120:0/66681182' entity='mgr.x' 2023-06-23T01:11:15.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:14 smithi120 ceph-mon[79813]: from='mgr.14120 172.21.15.120:0/66681182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:11:15.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:14 smithi120 ceph-mon[79813]: from='mgr.14120 172.21.15.120:0/66681182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:11:15.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:14 smithi120 ceph-mon[79813]: from='mgr.14120 172.21.15.120:0/66681182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:11:15.915 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:15 smithi120 ceph-mon[79813]: Found migration_current of "None". Setting to last migration. 2023-06-23T01:11:15.915 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:15 smithi120 ceph-mon[79813]: from='mgr.14120 172.21.15.120:0/66681182' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-06-23T01:11:15.915 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:15 smithi120 ceph-mon[79813]: from='mgr.14120 172.21.15.120:0/66681182' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-06-23T01:11:15.915 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:15 smithi120 ceph-mon[79813]: [23/Jun/2023:01:11:15] ENGINE Bus STARTING 2023-06-23T01:11:15.916 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:15 smithi120 ceph-mon[79813]: mgrmap e7: x(active, since 1.00694s) 2023-06-23T01:11:15.916 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:15 smithi120 ceph-mon[79813]: from='mgr.14120 172.21.15.120:0/66681182' entity='mgr.x' 2023-06-23T01:11:15.929 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout { 2023-06-23T01:11:15.934 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-06-23T01:11:15.934 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "initialized": true 2023-06-23T01:11:15.934 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout } 2023-06-23T01:11:15.934 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.550+0000 7f6a24467700 1 Processor -- start 2023-06-23T01:11:15.935 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.550+0000 7f6a24467700 1 -- start start 2023-06-23T01:11:15.935 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.550+0000 7f6a24467700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a1c102210 0x7f6a1c102620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:15.935 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.550+0000 7f6a24467700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1c102b60 con 0x7f6a1c102210 2023-06-23T01:11:15.935 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.551+0000 7f6a22203700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a1c102210 0x7f6a1c102620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:15.935 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.551+0000 7f6a22203700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a1c102210 0x7f6a1c102620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49468/0 (socket says 172.21.15.120:49468) 2023-06-23T01:11:15.935 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.551+0000 7f6a22203700 1 -- 172.21.15.120:0/4174422009 learned_addr learned my addr 172.21.15.120:0/4174422009 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:15.935 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.551+0000 7f6a22203700 1 -- 172.21.15.120:0/4174422009 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a1c102ca0 con 0x7f6a1c102210 2023-06-23T01:11:15.935 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.551+0000 7f6a22203700 1 --2- 172.21.15.120:0/4174422009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a1c102210 0x7f6a1c102620 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6a180098e0 tx=0x7f6a18009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc29ef4279ad865 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:15.936 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.552+0000 7f6a21201700 1 -- 172.21.15.120:0/4174422009 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6a18009e80 con 0x7f6a1c102210 2023-06-23T01:11:15.936 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.552+0000 7f6a21201700 1 -- 172.21.15.120:0/4174422009 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6a1800bd10 con 0x7f6a1c102210 2023-06-23T01:11:15.936 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.552+0000 7f6a21201700 1 -- 172.21.15.120:0/4174422009 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6a18011ce0 con 0x7f6a1c102210 2023-06-23T01:11:15.936 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.552+0000 7f6a24467700 1 -- 172.21.15.120:0/4174422009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a1c102210 msgr2=0x7f6a1c102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:15.936 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.552+0000 7f6a24467700 1 --2- 172.21.15.120:0/4174422009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a1c102210 0x7f6a1c102620 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6a180098e0 tx=0x7f6a18009250 comp rx=0 tx=0).stop 2023-06-23T01:11:15.936 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.552+0000 7f6a24467700 1 -- 172.21.15.120:0/4174422009 shutdown_connections 2023-06-23T01:11:15.936 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.552+0000 7f6a24467700 1 --2- 172.21.15.120:0/4174422009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a1c102210 0x7f6a1c102620 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:15.937 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.552+0000 7f6a24467700 1 -- 172.21.15.120:0/4174422009 >> 172.21.15.120:0/4174422009 conn(0x7f6a1c0fd990 msgr2=0x7f6a1c0ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:15.937 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.552+0000 7f6a24467700 1 -- 172.21.15.120:0/4174422009 shutdown_connections 2023-06-23T01:11:15.937 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.553+0000 7f6a24467700 1 -- 172.21.15.120:0/4174422009 wait complete. 2023-06-23T01:11:15.937 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.553+0000 7f6a24467700 1 Processor -- start 2023-06-23T01:11:15.937 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.553+0000 7f6a24467700 1 -- start start 2023-06-23T01:11:15.937 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.554+0000 7f6a24467700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a1c102210 0x7f6a1c1130f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:15.937 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.554+0000 7f6a24467700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1c113630 con 0x7f6a1c102210 2023-06-23T01:11:15.937 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.554+0000 7f6a22203700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a1c102210 0x7f6a1c1130f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:15.938 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.554+0000 7f6a22203700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a1c102210 0x7f6a1c1130f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49484/0 (socket says 172.21.15.120:49484) 2023-06-23T01:11:15.938 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.554+0000 7f6a22203700 1 -- 172.21.15.120:0/2289349866 learned_addr learned my addr 172.21.15.120:0/2289349866 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:15.938 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.554+0000 7f6a22203700 1 -- 172.21.15.120:0/2289349866 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a180094f0 con 0x7f6a1c102210 2023-06-23T01:11:15.938 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.555+0000 7f6a22203700 1 --2- 172.21.15.120:0/2289349866 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a1c102210 0x7f6a1c1130f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6a18006b00 tx=0x7f6a18009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:15.938 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.555+0000 7f6a12ffd700 1 -- 172.21.15.120:0/2289349866 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6a1801a640 con 0x7f6a1c102210 2023-06-23T01:11:15.938 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.555+0000 7f6a12ffd700 1 -- 172.21.15.120:0/2289349866 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6a1801ac50 con 0x7f6a1c102210 2023-06-23T01:11:15.938 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.555+0000 7f6a24467700 1 -- 172.21.15.120:0/2289349866 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a1c113830 con 0x7f6a1c102210 2023-06-23T01:11:15.939 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.555+0000 7f6a12ffd700 1 -- 172.21.15.120:0/2289349866 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6a18022bb0 con 0x7f6a1c102210 2023-06-23T01:11:15.939 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.555+0000 7f6a24467700 1 -- 172.21.15.120:0/2289349866 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a1c1144d0 con 0x7f6a1c102210 2023-06-23T01:11:15.939 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.556+0000 7f6a12ffd700 1 -- 172.21.15.120:0/2289349866 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) v1 ==== 43219+0+0 (secure 0 0 0) 0x7f6a18022490 con 0x7f6a1c102210 2023-06-23T01:11:15.939 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.557+0000 7f6a12ffd700 1 --2- 172.21.15.120:0/2289349866 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7f6a080365c0 0x7f6a08038a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:15.939 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.557+0000 7f6a12ffd700 1 -- 172.21.15.120:0/2289349866 --> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6a08039180 con 0x7f6a080365c0 2023-06-23T01:11:15.939 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.557+0000 7f6a12ffd700 1 -- 172.21.15.120:0/2289349866 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f6a1804a3b0 con 0x7f6a1c102210 2023-06-23T01:11:15.939 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.557+0000 7f6a21a02700 1 -- 172.21.15.120:0/2289349866 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7f6a080365c0 msgr2=0x7f6a08038a70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.120:6800/506843621 2023-06-23T01:11:15.939 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.557+0000 7f6a21a02700 1 --2- 172.21.15.120:0/2289349866 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7f6a080365c0 0x7f6a08038a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-23T01:11:15.940 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.758+0000 7f6a21a02700 1 -- 172.21.15.120:0/2289349866 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7f6a080365c0 msgr2=0x7f6a08038a70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.120:6800/506843621 2023-06-23T01:11:15.940 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:08.758+0000 7f6a21a02700 1 --2- 172.21.15.120:0/2289349866 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7f6a080365c0 0x7f6a08038a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-06-23T01:11:15.940 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:09.159+0000 7f6a21a02700 1 -- 172.21.15.120:0/2289349866 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7f6a080365c0 msgr2=0x7f6a08038a70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.120:6800/506843621 2023-06-23T01:11:15.940 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:09.159+0000 7f6a21a02700 1 --2- 172.21.15.120:0/2289349866 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7f6a080365c0 0x7f6a08038a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-06-23T01:11:15.940 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:09.960+0000 7f6a21a02700 1 -- 172.21.15.120:0/2289349866 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7f6a080365c0 msgr2=0x7f6a08038a70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.120:6800/506843621 2023-06-23T01:11:15.940 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:09.960+0000 7f6a21a02700 1 --2- 172.21.15.120:0/2289349866 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7f6a080365c0 0x7f6a08038a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-06-23T01:11:15.940 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:11.561+0000 7f6a21a02700 1 -- 172.21.15.120:0/2289349866 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7f6a080365c0 msgr2=0x7f6a08038a70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.120:6800/506843621 2023-06-23T01:11:15.941 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:11.562+0000 7f6a21a02700 1 --2- 172.21.15.120:0/2289349866 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7f6a080365c0 0x7f6a08038a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-06-23T01:11:15.941 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:14.577+0000 7f6a12ffd700 1 -- 172.21.15.120:0/2289349866 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mgrmap(e 6) v1 ==== 42986+0+0 (secure 0 0 0) 0x7f6a1801a7a0 con 0x7f6a1c102210 2023-06-23T01:11:15.941 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:14.577+0000 7f6a12ffd700 1 -- 172.21.15.120:0/2289349866 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7f6a080365c0 msgr2=0x7f6a08038a70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-23T01:11:15.941 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:14.577+0000 7f6a12ffd700 1 --2- 172.21.15.120:0/2289349866 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7f6a080365c0 0x7f6a08038a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:15.941 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:15.579+0000 7f6a12ffd700 1 -- 172.21.15.120:0/2289349866 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mgrmap(e 7) v1 ==== 43113+0+0 (secure 0 0 0) 0x7f6a1802c440 con 0x7f6a1c102210 2023-06-23T01:11:15.941 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:15.579+0000 7f6a12ffd700 1 --2- 172.21.15.120:0/2289349866 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f6a08039e90 0x7f6a0803c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:15.941 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:15.579+0000 7f6a12ffd700 1 -- 172.21.15.120:0/2289349866 --> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6a08039180 con 0x7f6a08039e90 2023-06-23T01:11:15.941 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:15.579+0000 7f6a21a02700 1 --2- 172.21.15.120:0/2289349866 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f6a08039e90 0x7f6a0803c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:15.942 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:15.580+0000 7f6a21a02700 1 --2- 172.21.15.120:0/2289349866 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f6a08039e90 0x7f6a0803c270 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c0039d0 tx=0x7f6a0c009350 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:15.942 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:15.581+0000 7f6a12ffd700 1 -- 172.21.15.120:0/2289349866 <== mgr.14120 v2:172.21.15.120:6800/1836713308 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f6a08039180 con 0x7f6a08039e90 2023-06-23T01:11:15.942 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:15.587+0000 7f6a24467700 1 -- 172.21.15.120:0/2289349866 --> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f6a1c113cc0 con 0x7f6a08039e90 2023-06-23T01:11:15.942 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:15.587+0000 7f6a12ffd700 1 -- 172.21.15.120:0/2289349866 <== mgr.14120 v2:172.21.15.120:6800/1836713308 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f6a1c113cc0 con 0x7f6a08039e90 2023-06-23T01:11:15.942 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:15.588+0000 7f6a24467700 1 -- 172.21.15.120:0/2289349866 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f6a08039e90 msgr2=0x7f6a0803c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:15.942 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:15.588+0000 7f6a24467700 1 --2- 172.21.15.120:0/2289349866 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f6a08039e90 0x7f6a0803c270 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c0039d0 tx=0x7f6a0c009350 comp rx=0 tx=0).stop 2023-06-23T01:11:15.942 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:15.588+0000 7f6a24467700 1 -- 172.21.15.120:0/2289349866 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a1c102210 msgr2=0x7f6a1c1130f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:15.942 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:15.588+0000 7f6a24467700 1 --2- 172.21.15.120:0/2289349866 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a1c102210 0x7f6a1c1130f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6a18006b00 tx=0x7f6a18009250 comp rx=0 tx=0).stop 2023-06-23T01:11:15.943 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:15.588+0000 7f6a24467700 1 -- 172.21.15.120:0/2289349866 shutdown_connections 2023-06-23T01:11:15.943 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:15.588+0000 7f6a24467700 1 --2- 172.21.15.120:0/2289349866 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f6a08039e90 0x7f6a0803c270 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:15.943 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:15.588+0000 7f6a24467700 1 --2- 172.21.15.120:0/2289349866 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a1c102210 0x7f6a1c1130f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:15.943 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:15.588+0000 7f6a24467700 1 --2- 172.21.15.120:0/2289349866 >> [v2:172.21.15.120:6800/506843621,v1:172.21.15.120:6801/506843621] conn(0x7f6a080365c0 0x7f6a08038a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:15.943 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:15.588+0000 7f6a24467700 1 -- 172.21.15.120:0/2289349866 >> 172.21.15.120:0/2289349866 conn(0x7f6a1c0fd990 msgr2=0x7f6a1c0fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:15.943 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:15.588+0000 7f6a24467700 1 -- 172.21.15.120:0/2289349866 shutdown_connections 2023-06-23T01:11:15.943 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:15.588+0000 7f6a24467700 1 -- 172.21.15.120:0/2289349866 wait complete. 2023-06-23T01:11:15.944 INFO:teuthology.orchestra.run.smithi120.stdout:mgr epoch 5 is available 2023-06-23T01:11:15.944 INFO:teuthology.orchestra.run.smithi120.stdout:Setting orchestrator backend to cephadm... 2023-06-23T01:11:17.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:16 smithi120 ceph-mon[79813]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-06-23T01:11:17.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:16 smithi120 ceph-mon[79813]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-06-23T01:11:17.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:16 smithi120 ceph-mon[79813]: [23/Jun/2023:01:11:15] ENGINE Serving on https://172.21.15.120:7150 2023-06-23T01:11:17.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:16 smithi120 ceph-mon[79813]: [23/Jun/2023:01:11:15] ENGINE Bus STARTED 2023-06-23T01:11:17.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:16 smithi120 ceph-mon[79813]: from='mgr.14120 172.21.15.120:0/66681182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:11:17.107 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.433+0000 7f03a1177700 1 Processor -- start 2023-06-23T01:11:17.107 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.434+0000 7f03a1177700 1 -- start start 2023-06-23T01:11:17.107 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.434+0000 7f03a1177700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f039c070e10 0x7f039c104ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:17.108 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.434+0000 7f03a1177700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039c1053e0 con 0x7f039c070e10 2023-06-23T01:11:17.108 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.434+0000 7f039ad9d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f039c070e10 0x7f039c104ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:17.108 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.434+0000 7f039ad9d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f039c070e10 0x7f039c104ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42046/0 (socket says 172.21.15.120:42046) 2023-06-23T01:11:17.109 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.434+0000 7f039ad9d700 1 -- 172.21.15.120:0/2496605809 learned_addr learned my addr 172.21.15.120:0/2496605809 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:17.109 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.435+0000 7f039ad9d700 1 -- 172.21.15.120:0/2496605809 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f039c105520 con 0x7f039c070e10 2023-06-23T01:11:17.109 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.435+0000 7f039ad9d700 1 --2- 172.21.15.120:0/2496605809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f039c070e10 0x7f039c104ea0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f038c0098e0 tx=0x7f038c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2132a5c503785f4c server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:17.109 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.435+0000 7f039a59c700 1 -- 172.21.15.120:0/2496605809 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f038c009e80 con 0x7f039c070e10 2023-06-23T01:11:17.110 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.435+0000 7f039a59c700 1 -- 172.21.15.120:0/2496605809 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f038c00bd10 con 0x7f039c070e10 2023-06-23T01:11:17.110 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.435+0000 7f039a59c700 1 -- 172.21.15.120:0/2496605809 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f038c011ce0 con 0x7f039c070e10 2023-06-23T01:11:17.110 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.436+0000 7f03a1177700 1 -- 172.21.15.120:0/2496605809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f039c070e10 msgr2=0x7f039c104ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:17.110 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.436+0000 7f03a1177700 1 --2- 172.21.15.120:0/2496605809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f039c070e10 0x7f039c104ea0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f038c0098e0 tx=0x7f038c009250 comp rx=0 tx=0).stop 2023-06-23T01:11:17.110 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.436+0000 7f03a1177700 1 -- 172.21.15.120:0/2496605809 shutdown_connections 2023-06-23T01:11:17.111 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.436+0000 7f03a1177700 1 --2- 172.21.15.120:0/2496605809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f039c070e10 0x7f039c104ea0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:17.111 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.436+0000 7f03a1177700 1 -- 172.21.15.120:0/2496605809 >> 172.21.15.120:0/2496605809 conn(0x7f039c0fc8f0 msgr2=0x7f039c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:17.111 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.436+0000 7f03a1177700 1 -- 172.21.15.120:0/2496605809 shutdown_connections 2023-06-23T01:11:17.111 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.436+0000 7f03a1177700 1 -- 172.21.15.120:0/2496605809 wait complete. 2023-06-23T01:11:17.111 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.437+0000 7f03a1177700 1 Processor -- start 2023-06-23T01:11:17.111 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.437+0000 7f03a1177700 1 -- start start 2023-06-23T01:11:17.111 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.437+0000 7f03a1177700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f039c070e10 0x7f039c110f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:17.112 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.437+0000 7f03a1177700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039c111440 con 0x7f039c070e10 2023-06-23T01:11:17.112 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.437+0000 7f039ad9d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f039c070e10 0x7f039c110f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:17.112 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.437+0000 7f039ad9d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f039c070e10 0x7f039c110f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42052/0 (socket says 172.21.15.120:42052) 2023-06-23T01:11:17.112 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.437+0000 7f039ad9d700 1 -- 172.21.15.120:0/841336943 learned_addr learned my addr 172.21.15.120:0/841336943 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:17.112 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.438+0000 7f039ad9d700 1 -- 172.21.15.120:0/841336943 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f038c0094f0 con 0x7f039c070e10 2023-06-23T01:11:17.113 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.438+0000 7f039ad9d700 1 --2- 172.21.15.120:0/841336943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f039c070e10 0x7f039c110f00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f038c011650 tx=0x7f038c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:17.113 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.438+0000 7f03937fe700 1 -- 172.21.15.120:0/841336943 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f038c01a640 con 0x7f039c070e10 2023-06-23T01:11:17.113 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.438+0000 7f03937fe700 1 -- 172.21.15.120:0/841336943 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f038c01ac50 con 0x7f039c070e10 2023-06-23T01:11:17.113 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.438+0000 7f03a1177700 1 -- 172.21.15.120:0/841336943 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f039c111640 con 0x7f039c070e10 2023-06-23T01:11:17.113 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.438+0000 7f03937fe700 1 -- 172.21.15.120:0/841336943 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f038c022b20 con 0x7f039c070e10 2023-06-23T01:11:17.113 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.438+0000 7f03a1177700 1 -- 172.21.15.120:0/841336943 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f039c1122e0 con 0x7f039c070e10 2023-06-23T01:11:17.113 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.439+0000 7f03937fe700 1 -- 172.21.15.120:0/841336943 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 7) v1 ==== 43113+0+0 (secure 0 0 0) 0x7f038c022400 con 0x7f039c070e10 2023-06-23T01:11:17.114 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.439+0000 7f03a1177700 1 -- 172.21.15.120:0/841336943 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f039c05f060 con 0x7f039c070e10 2023-06-23T01:11:17.114 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.440+0000 7f03937fe700 1 --2- 172.21.15.120:0/841336943 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f03880364a0 0x7f0388038950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:17.114 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.440+0000 7f03937fe700 1 -- 172.21.15.120:0/841336943 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f038c04a2c0 con 0x7f039c070e10 2023-06-23T01:11:17.114 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.440+0000 7f0393fff700 1 --2- 172.21.15.120:0/841336943 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f03880364a0 0x7f0388038950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:17.114 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.441+0000 7f0393fff700 1 --2- 172.21.15.120:0/841336943 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f03880364a0 0x7f0388038950 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0384009990 tx=0x7f0384006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:17.114 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.445+0000 7f03937fe700 1 -- 172.21.15.120:0/841336943 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f038c01adc0 con 0x7f039c070e10 2023-06-23T01:11:17.114 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.629+0000 7f03937fe700 1 -- 172.21.15.120:0/841336943 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mgrmap(e 8) v1 ==== 43219+0+0 (secure 0 0 0) 0x7f038c02c960 con 0x7f039c070e10 2023-06-23T01:11:17.115 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.652+0000 7f03a1177700 1 -- 172.21.15.120:0/841336943 --> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f039c0fdd60 con 0x7f03880364a0 2023-06-23T01:11:17.115 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.658+0000 7f03937fe700 1 -- 172.21.15.120:0/841336943 <== mgr.14120 v2:172.21.15.120:6800/1836713308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f039c0fdd60 con 0x7f03880364a0 2023-06-23T01:11:17.115 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.661+0000 7f03a1177700 1 -- 172.21.15.120:0/841336943 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f03880364a0 msgr2=0x7f0388038950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:17.115 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.661+0000 7f03a1177700 1 --2- 172.21.15.120:0/841336943 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f03880364a0 0x7f0388038950 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0384009990 tx=0x7f0384006e40 comp rx=0 tx=0).stop 2023-06-23T01:11:17.115 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.661+0000 7f03a1177700 1 -- 172.21.15.120:0/841336943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f039c070e10 msgr2=0x7f039c110f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:17.115 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.661+0000 7f03a1177700 1 --2- 172.21.15.120:0/841336943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f039c070e10 0x7f039c110f00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f038c011650 tx=0x7f038c009250 comp rx=0 tx=0).stop 2023-06-23T01:11:17.115 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.661+0000 7f03a1177700 1 -- 172.21.15.120:0/841336943 shutdown_connections 2023-06-23T01:11:17.116 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.661+0000 7f03a1177700 1 --2- 172.21.15.120:0/841336943 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f03880364a0 0x7f0388038950 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:17.116 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.661+0000 7f03a1177700 1 --2- 172.21.15.120:0/841336943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f039c070e10 0x7f039c110f00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:17.116 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.661+0000 7f03a1177700 1 -- 172.21.15.120:0/841336943 >> 172.21.15.120:0/841336943 conn(0x7f039c0fc8f0 msgr2=0x7f039c0fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:17.116 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.661+0000 7f03a1177700 1 -- 172.21.15.120:0/841336943 shutdown_connections 2023-06-23T01:11:17.116 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:16.661+0000 7f03a1177700 1 -- 172.21.15.120:0/841336943 wait complete. 2023-06-23T01:11:18.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:17 smithi120 ceph-mon[79813]: mgrmap e8: x(active, since 2s) 2023-06-23T01:11:18.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:17 smithi120 ceph-mon[79813]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:11:18.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:17 smithi120 ceph-mon[79813]: from='mgr.14120 172.21.15.120:0/66681182' entity='mgr.x' 2023-06-23T01:11:18.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:17 smithi120 ceph-mon[79813]: from='mgr.14120 172.21.15.120:0/66681182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:11:18.377 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout value unchanged 2023-06-23T01:11:18.377 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.807+0000 7fb27c3f0700 1 Processor -- start 2023-06-23T01:11:18.377 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.807+0000 7fb27c3f0700 1 -- start start 2023-06-23T01:11:18.377 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.807+0000 7fb27c3f0700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb274077c60 0x7fb274076170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:18.377 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.807+0000 7fb27c3f0700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb274078070 con 0x7fb274077c60 2023-06-23T01:11:18.378 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.808+0000 7fb27a18c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb274077c60 0x7fb274076170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:18.378 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.808+0000 7fb27a18c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb274077c60 0x7fb274076170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42054/0 (socket says 172.21.15.120:42054) 2023-06-23T01:11:18.378 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.808+0000 7fb27a18c700 1 -- 172.21.15.120:0/3633857381 learned_addr learned my addr 172.21.15.120:0/3633857381 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:18.378 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.808+0000 7fb27a18c700 1 -- 172.21.15.120:0/3633857381 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2740766b0 con 0x7fb274077c60 2023-06-23T01:11:18.378 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.808+0000 7fb27a18c700 1 --2- 172.21.15.120:0/3633857381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb274077c60 0x7fb274076170 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb2700098e0 tx=0x7fb270009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0c08d27ffe7fe3d server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:18.378 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.809+0000 7fb27918a700 1 -- 172.21.15.120:0/3633857381 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb270009e80 con 0x7fb274077c60 2023-06-23T01:11:18.378 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.809+0000 7fb27918a700 1 -- 172.21.15.120:0/3633857381 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb27000bd10 con 0x7fb274077c60 2023-06-23T01:11:18.378 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.809+0000 7fb27918a700 1 -- 172.21.15.120:0/3633857381 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb270011ce0 con 0x7fb274077c60 2023-06-23T01:11:18.379 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.809+0000 7fb27c3f0700 1 -- 172.21.15.120:0/3633857381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb274077c60 msgr2=0x7fb274076170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:18.379 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.809+0000 7fb27c3f0700 1 --2- 172.21.15.120:0/3633857381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb274077c60 0x7fb274076170 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb2700098e0 tx=0x7fb270009250 comp rx=0 tx=0).stop 2023-06-23T01:11:18.379 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.809+0000 7fb27c3f0700 1 -- 172.21.15.120:0/3633857381 shutdown_connections 2023-06-23T01:11:18.379 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.809+0000 7fb27c3f0700 1 --2- 172.21.15.120:0/3633857381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb274077c60 0x7fb274076170 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:18.379 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.809+0000 7fb27c3f0700 1 -- 172.21.15.120:0/3633857381 >> 172.21.15.120:0/3633857381 conn(0x7fb2740fd9f0 msgr2=0x7fb2740ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:18.379 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.809+0000 7fb27c3f0700 1 -- 172.21.15.120:0/3633857381 shutdown_connections 2023-06-23T01:11:18.379 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.809+0000 7fb27c3f0700 1 -- 172.21.15.120:0/3633857381 wait complete. 2023-06-23T01:11:18.380 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.810+0000 7fb27c3f0700 1 Processor -- start 2023-06-23T01:11:18.380 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.810+0000 7fb27c3f0700 1 -- start start 2023-06-23T01:11:18.380 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.811+0000 7fb27c3f0700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb274077c60 0x7fb2741130c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:18.380 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.811+0000 7fb27c3f0700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb274113600 con 0x7fb274077c60 2023-06-23T01:11:18.380 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.811+0000 7fb27a18c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb274077c60 0x7fb2741130c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:18.380 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.811+0000 7fb27a18c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb274077c60 0x7fb2741130c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42062/0 (socket says 172.21.15.120:42062) 2023-06-23T01:11:18.380 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.811+0000 7fb27a18c700 1 -- 172.21.15.120:0/3787912411 learned_addr learned my addr 172.21.15.120:0/3787912411 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:18.380 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.811+0000 7fb27a18c700 1 -- 172.21.15.120:0/3787912411 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2700094f0 con 0x7fb274077c60 2023-06-23T01:11:18.381 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.811+0000 7fb27a18c700 1 --2- 172.21.15.120:0/3787912411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb274077c60 0x7fb2741130c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb270011650 tx=0x7fb270009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:18.381 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.811+0000 7fb26affd700 1 -- 172.21.15.120:0/3787912411 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb27001a640 con 0x7fb274077c60 2023-06-23T01:11:18.381 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.812+0000 7fb26affd700 1 -- 172.21.15.120:0/3787912411 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb27001ac50 con 0x7fb274077c60 2023-06-23T01:11:18.381 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.812+0000 7fb27c3f0700 1 -- 172.21.15.120:0/3787912411 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb274113800 con 0x7fb274077c60 2023-06-23T01:11:18.381 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.812+0000 7fb26affd700 1 -- 172.21.15.120:0/3787912411 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb270022b20 con 0x7fb274077c60 2023-06-23T01:11:18.381 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.812+0000 7fb27c3f0700 1 -- 172.21.15.120:0/3787912411 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2741144a0 con 0x7fb274077c60 2023-06-23T01:11:18.381 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.813+0000 7fb26affd700 1 -- 172.21.15.120:0/3787912411 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 8) v1 ==== 43219+0+0 (secure 0 0 0) 0x7fb270022400 con 0x7fb274077c60 2023-06-23T01:11:18.381 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.813+0000 7fb27c3f0700 1 -- 172.21.15.120:0/3787912411 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb27405f060 con 0x7fb274077c60 2023-06-23T01:11:18.382 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.813+0000 7fb26affd700 1 --2- 172.21.15.120:0/3787912411 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7fb2600365c0 0x7fb260038a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:18.382 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.813+0000 7fb26affd700 1 -- 172.21.15.120:0/3787912411 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fb27004a4d0 con 0x7fb274077c60 2023-06-23T01:11:18.382 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.813+0000 7fb27998b700 1 --2- 172.21.15.120:0/3787912411 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7fb2600365c0 0x7fb260038a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:18.382 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.814+0000 7fb27998b700 1 --2- 172.21.15.120:0/3787912411 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7fb2600365c0 0x7fb260038a70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb264009990 tx=0x7fb264006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:18.382 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:17.819+0000 7fb26affd700 1 -- 172.21.15.120:0/3787912411 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fb27001adc0 con 0x7fb274077c60 2023-06-23T01:11:18.382 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.011+0000 7fb27c3f0700 1 -- 172.21.15.120:0/3787912411 --> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2740feda0 con 0x7fb2600365c0 2023-06-23T01:11:18.382 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.011+0000 7fb26affd700 1 -- 172.21.15.120:0/3787912411 <== mgr.14120 v2:172.21.15.120:6800/1836713308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fb2740feda0 con 0x7fb2600365c0 2023-06-23T01:11:18.383 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.014+0000 7fb27c3f0700 1 -- 172.21.15.120:0/3787912411 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7fb2600365c0 msgr2=0x7fb260038a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:18.383 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.014+0000 7fb27c3f0700 1 --2- 172.21.15.120:0/3787912411 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7fb2600365c0 0x7fb260038a70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb264009990 tx=0x7fb264006e40 comp rx=0 tx=0).stop 2023-06-23T01:11:18.383 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.014+0000 7fb27c3f0700 1 -- 172.21.15.120:0/3787912411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb274077c60 msgr2=0x7fb2741130c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:18.383 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.014+0000 7fb27c3f0700 1 --2- 172.21.15.120:0/3787912411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb274077c60 0x7fb2741130c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb270011650 tx=0x7fb270009250 comp rx=0 tx=0).stop 2023-06-23T01:11:18.383 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.014+0000 7fb27c3f0700 1 -- 172.21.15.120:0/3787912411 shutdown_connections 2023-06-23T01:11:18.383 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.014+0000 7fb27c3f0700 1 --2- 172.21.15.120:0/3787912411 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7fb2600365c0 0x7fb260038a70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:18.383 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.014+0000 7fb27c3f0700 1 --2- 172.21.15.120:0/3787912411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb274077c60 0x7fb2741130c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:18.383 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.014+0000 7fb27c3f0700 1 -- 172.21.15.120:0/3787912411 >> 172.21.15.120:0/3787912411 conn(0x7fb2740fd9f0 msgr2=0x7fb2740fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:18.384 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.014+0000 7fb27c3f0700 1 -- 172.21.15.120:0/3787912411 shutdown_connections 2023-06-23T01:11:18.384 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.014+0000 7fb27c3f0700 1 -- 172.21.15.120:0/3787912411 wait complete. 2023-06-23T01:11:18.384 INFO:teuthology.orchestra.run.smithi120.stdout:Generating ssh key... 2023-06-23T01:11:19.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:18 smithi120 ceph-mon[79813]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:11:19.510 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:19 smithi120 conmon[80224]: Generating public/private rsa key pair. 2023-06-23T01:11:19.510 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:19 smithi120 conmon[80224]: Your identification has been saved in /tmp/tmptvwlg7uj/key. 2023-06-23T01:11:19.510 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:19 smithi120 conmon[80224]: Your public key has been saved in /tmp/tmptvwlg7uj/key.pub. 2023-06-23T01:11:19.511 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:19 smithi120 conmon[80224]: The key fingerprint is: 2023-06-23T01:11:19.511 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:19 smithi120 conmon[80224]: SHA256:u+5PDDXVlfw9Sk4kR70zYUe6EHpUCMPfIvBW3fxCEfU ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c 2023-06-23T01:11:19.511 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:19 smithi120 conmon[80224]: The key's randomart image is: 2023-06-23T01:11:19.511 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:19 smithi120 conmon[80224]: +---[RSA 3072]----+ 2023-06-23T01:11:19.511 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:19 smithi120 conmon[80224]: | .o.=**O=| 2023-06-23T01:11:19.511 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:19 smithi120 conmon[80224]: | . .*o++B=| 2023-06-23T01:11:19.511 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:19 smithi120 conmon[80224]: | o+o*oo E| 2023-06-23T01:11:19.512 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:19 smithi120 conmon[80224]: | .+oo+o*+| 2023-06-23T01:11:19.512 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:19 smithi120 conmon[80224]: | S. .+.o.+| 2023-06-23T01:11:19.512 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:19 smithi120 conmon[80224]: | + o | 2023-06-23T01:11:19.512 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:19 smithi120 conmon[80224]: | . o | 2023-06-23T01:11:19.512 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:19 smithi120 conmon[80224]: | o | 2023-06-23T01:11:19.512 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:19 smithi120 conmon[80224]: | o+.. | 2023-06-23T01:11:19.513 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:19 smithi120 conmon[80224]: +----[SHA256]-----+ 2023-06-23T01:11:19.647 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.880+0000 7fdfb9557700 1 Processor -- start 2023-06-23T01:11:19.647 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.880+0000 7fdfb9557700 1 -- start start 2023-06-23T01:11:19.647 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.880+0000 7fdfb9557700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfb4077c90 0x7fdfb4076130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:19.648 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.880+0000 7fdfb9557700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb40780a0 con 0x7fdfb4077c90 2023-06-23T01:11:19.648 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.881+0000 7fdfb2ffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfb4077c90 0x7fdfb4076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:19.648 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.881+0000 7fdfb2ffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfb4077c90 0x7fdfb4076130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42072/0 (socket says 172.21.15.120:42072) 2023-06-23T01:11:19.648 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.881+0000 7fdfb2ffd700 1 -- 172.21.15.120:0/3263584296 learned_addr learned my addr 172.21.15.120:0/3263584296 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:19.648 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.881+0000 7fdfb2ffd700 1 -- 172.21.15.120:0/3263584296 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb4076670 con 0x7fdfb4077c90 2023-06-23T01:11:19.649 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.881+0000 7fdfb2ffd700 1 --2- 172.21.15.120:0/3263584296 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfb4077c90 0x7fdfb4076130 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdfa40098e0 tx=0x7fdfa4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b5ebe4ae4c4c700 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:19.649 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.881+0000 7fdfb1ffb700 1 -- 172.21.15.120:0/3263584296 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdfa4009e80 con 0x7fdfb4077c90 2023-06-23T01:11:19.649 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.881+0000 7fdfb1ffb700 1 -- 172.21.15.120:0/3263584296 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdfa400bd10 con 0x7fdfb4077c90 2023-06-23T01:11:19.649 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.881+0000 7fdfb1ffb700 1 -- 172.21.15.120:0/3263584296 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdfa4011ce0 con 0x7fdfb4077c90 2023-06-23T01:11:19.649 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.882+0000 7fdfb9557700 1 -- 172.21.15.120:0/3263584296 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfb4077c90 msgr2=0x7fdfb4076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:19.649 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.882+0000 7fdfb9557700 1 --2- 172.21.15.120:0/3263584296 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfb4077c90 0x7fdfb4076130 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdfa40098e0 tx=0x7fdfa4009250 comp rx=0 tx=0).stop 2023-06-23T01:11:19.649 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.882+0000 7fdfb9557700 1 -- 172.21.15.120:0/3263584296 shutdown_connections 2023-06-23T01:11:19.650 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.882+0000 7fdfb9557700 1 --2- 172.21.15.120:0/3263584296 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfb4077c90 0x7fdfb4076130 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:19.650 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.882+0000 7fdfb9557700 1 -- 172.21.15.120:0/3263584296 >> 172.21.15.120:0/3263584296 conn(0x7fdfb40fd990 msgr2=0x7fdfb40ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:19.650 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.882+0000 7fdfb9557700 1 -- 172.21.15.120:0/3263584296 shutdown_connections 2023-06-23T01:11:19.650 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.882+0000 7fdfb9557700 1 -- 172.21.15.120:0/3263584296 wait complete. 2023-06-23T01:11:19.650 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.883+0000 7fdfb9557700 1 Processor -- start 2023-06-23T01:11:19.650 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.883+0000 7fdfb9557700 1 -- start start 2023-06-23T01:11:19.651 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.883+0000 7fdfb9557700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfb4077c90 0x7fdfb4113100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:19.651 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.883+0000 7fdfb9557700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb4113640 con 0x7fdfb4077c90 2023-06-23T01:11:19.651 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.884+0000 7fdfb2ffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfb4077c90 0x7fdfb4113100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:19.651 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.884+0000 7fdfb2ffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfb4077c90 0x7fdfb4113100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42086/0 (socket says 172.21.15.120:42086) 2023-06-23T01:11:19.651 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.884+0000 7fdfb2ffd700 1 -- 172.21.15.120:0/2101355798 learned_addr learned my addr 172.21.15.120:0/2101355798 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:19.651 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.884+0000 7fdfb2ffd700 1 -- 172.21.15.120:0/2101355798 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa40094f0 con 0x7fdfb4077c90 2023-06-23T01:11:19.651 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.884+0000 7fdfb2ffd700 1 --2- 172.21.15.120:0/2101355798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfb4077c90 0x7fdfb4113100 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4011970 tx=0x7fdfa4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:19.652 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.884+0000 7fdf9b7fe700 1 -- 172.21.15.120:0/2101355798 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdfa4009e80 con 0x7fdfb4077c90 2023-06-23T01:11:19.652 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.884+0000 7fdf9b7fe700 1 -- 172.21.15.120:0/2101355798 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdfa401aaf0 con 0x7fdfb4077c90 2023-06-23T01:11:19.652 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.884+0000 7fdfb9557700 1 -- 172.21.15.120:0/2101355798 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfb4113840 con 0x7fdfb4077c90 2023-06-23T01:11:19.652 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.884+0000 7fdf9b7fe700 1 -- 172.21.15.120:0/2101355798 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdfa4022a90 con 0x7fdfb4077c90 2023-06-23T01:11:19.652 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.885+0000 7fdfb9557700 1 -- 172.21.15.120:0/2101355798 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfb41144e0 con 0x7fdfb4077c90 2023-06-23T01:11:19.652 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.885+0000 7fdf9b7fe700 1 -- 172.21.15.120:0/2101355798 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 8) v1 ==== 43219+0+0 (secure 0 0 0) 0x7fdfa4022400 con 0x7fdfb4077c90 2023-06-23T01:11:19.653 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.885+0000 7fdfb9557700 1 -- 172.21.15.120:0/2101355798 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfb405f060 con 0x7fdfb4077c90 2023-06-23T01:11:19.653 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.886+0000 7fdf9b7fe700 1 --2- 172.21.15.120:0/2101355798 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7fdf9c0365c0 0x7fdf9c038a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:19.653 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.886+0000 7fdf9b7fe700 1 -- 172.21.15.120:0/2101355798 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fdfa404a440 con 0x7fdfb4077c90 2023-06-23T01:11:19.653 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.886+0000 7fdfb27fc700 1 --2- 172.21.15.120:0/2101355798 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7fdf9c0365c0 0x7fdf9c038a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:19.653 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.887+0000 7fdfb27fc700 1 --2- 172.21.15.120:0/2101355798 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7fdf9c0365c0 0x7fdf9c038a70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8009990 tx=0x7fdfa8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:19.654 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:18.891+0000 7fdf9b7fe700 1 -- 172.21.15.120:0/2101355798 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fdfa4006d70 con 0x7fdfb4077c90 2023-06-23T01:11:19.655 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:19.080+0000 7fdfb9557700 1 -- 172.21.15.120:0/2101355798 --> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fdfb40fed20 con 0x7fdf9c0365c0 2023-06-23T01:11:19.655 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:19.258+0000 7fdf9b7fe700 1 -- 172.21.15.120:0/2101355798 <== mgr.14120 v2:172.21.15.120:6800/1836713308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fdfb40fed20 con 0x7fdf9c0365c0 2023-06-23T01:11:19.655 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:19.260+0000 7fdfb9557700 1 -- 172.21.15.120:0/2101355798 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7fdf9c0365c0 msgr2=0x7fdf9c038a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:19.655 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:19.261+0000 7fdfb9557700 1 --2- 172.21.15.120:0/2101355798 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7fdf9c0365c0 0x7fdf9c038a70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8009990 tx=0x7fdfa8006e40 comp rx=0 tx=0).stop 2023-06-23T01:11:19.655 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:19.261+0000 7fdfb9557700 1 -- 172.21.15.120:0/2101355798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfb4077c90 msgr2=0x7fdfb4113100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:19.655 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:19.261+0000 7fdfb9557700 1 --2- 172.21.15.120:0/2101355798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfb4077c90 0x7fdfb4113100 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4011970 tx=0x7fdfa4009250 comp rx=0 tx=0).stop 2023-06-23T01:11:19.656 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:19.261+0000 7fdfb9557700 1 -- 172.21.15.120:0/2101355798 shutdown_connections 2023-06-23T01:11:19.656 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:19.261+0000 7fdfb9557700 1 --2- 172.21.15.120:0/2101355798 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7fdf9c0365c0 0x7fdf9c038a70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:19.656 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:19.261+0000 7fdfb9557700 1 --2- 172.21.15.120:0/2101355798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfb4077c90 0x7fdfb4113100 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:19.656 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:19.261+0000 7fdfb9557700 1 -- 172.21.15.120:0/2101355798 >> 172.21.15.120:0/2101355798 conn(0x7fdfb40fd990 msgr2=0x7fdfb40fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:19.656 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:19.261+0000 7fdfb9557700 1 -- 172.21.15.120:0/2101355798 shutdown_connections 2023-06-23T01:11:19.656 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:19.261+0000 7fdfb9557700 1 -- 172.21.15.120:0/2101355798 wait complete. 2023-06-23T01:11:20.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:20 smithi120 ceph-mon[79813]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:11:20.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:20 smithi120 ceph-mon[79813]: Generating ssh key... 2023-06-23T01:11:20.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:20 smithi120 ceph-mon[79813]: from='mgr.14120 172.21.15.120:0/66681182' entity='mgr.x' 2023-06-23T01:11:20.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:20 smithi120 ceph-mon[79813]: from='mgr.14120 172.21.15.120:0/66681182' entity='mgr.x' 2023-06-23T01:11:20.820 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD3ZA/wTqrFlLLL8uTbViox16bCHCPhvYHVeyT6FEowkewIzpFqCqQg0Zubjs/5YVA1Vz4Pw2urX6BA7T2+Ecl+luB5xuBM9qMkgzOuKZvSBaswNrgKpvD7S/JM/uNDIOx+lmAtqsS2OaBIw649paJh064pbnw2PHqpju6qNzi+ly4MFDdadGupraQ48EV7BNSsGVBGAvE0QjpAnASqvSNDTwW29qkZ3x1zz377mPh3UUCA8JJ3GClcGxKO6skDcnKHB9NvnALdq8Bzg/Hc7YfpBXFEE1stdfRnVTFahaNHc5s9JhN/6E100CXNckT8Ig4spLi+02zcYe+FwzAMuLXRJMO/B30JsFpgihbmpxs72FuYgxN9XcgTOpOhoeB/PEksa/X3ww+LpfuMH/2dOFFhDsr4UR+nY7jNWr+TI126/sNzbsK7nkPhs4xra7Xk8M/uD7B3GOU2viql0W+cOd4hNx43rkStIFPJMbghDefNGccZVfxTdvyK1kI+WyXL1VE= ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c 2023-06-23T01:11:20.821 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.204+0000 7f0bd83c2700 1 Processor -- start 2023-06-23T01:11:20.821 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.204+0000 7f0bd83c2700 1 -- start start 2023-06-23T01:11:20.821 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.205+0000 7f0bd83c2700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0bd0104470 0x7f0bd0104880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:20.822 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.205+0000 7f0bd83c2700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd0104dc0 con 0x7f0bd0104470 2023-06-23T01:11:20.822 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.205+0000 7f0bd615e700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0bd0104470 0x7f0bd0104880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:20.822 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.205+0000 7f0bd615e700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0bd0104470 0x7f0bd0104880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42088/0 (socket says 172.21.15.120:42088) 2023-06-23T01:11:20.822 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.205+0000 7f0bd615e700 1 -- 172.21.15.120:0/3768204274 learned_addr learned my addr 172.21.15.120:0/3768204274 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:20.823 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.205+0000 7f0bd615e700 1 -- 172.21.15.120:0/3768204274 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd0104f00 con 0x7f0bd0104470 2023-06-23T01:11:20.823 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.206+0000 7f0bd615e700 1 --2- 172.21.15.120:0/3768204274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0bd0104470 0x7f0bd0104880 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc0098e0 tx=0x7f0bcc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ade7b277d6ae8e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:20.823 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.206+0000 7f0bd515c700 1 -- 172.21.15.120:0/3768204274 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0bcc009e80 con 0x7f0bd0104470 2023-06-23T01:11:20.823 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.206+0000 7f0bd515c700 1 -- 172.21.15.120:0/3768204274 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0bcc00bd10 con 0x7f0bd0104470 2023-06-23T01:11:20.823 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.206+0000 7f0bd515c700 1 -- 172.21.15.120:0/3768204274 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0bcc011ce0 con 0x7f0bd0104470 2023-06-23T01:11:20.823 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.206+0000 7f0bd83c2700 1 -- 172.21.15.120:0/3768204274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0bd0104470 msgr2=0x7f0bd0104880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:20.824 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.206+0000 7f0bd83c2700 1 --2- 172.21.15.120:0/3768204274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0bd0104470 0x7f0bd0104880 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc0098e0 tx=0x7f0bcc009250 comp rx=0 tx=0).stop 2023-06-23T01:11:20.824 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.207+0000 7f0bd83c2700 1 -- 172.21.15.120:0/3768204274 shutdown_connections 2023-06-23T01:11:20.824 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.207+0000 7f0bd83c2700 1 --2- 172.21.15.120:0/3768204274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0bd0104470 0x7f0bd0104880 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:20.824 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.207+0000 7f0bd83c2700 1 -- 172.21.15.120:0/3768204274 >> 172.21.15.120:0/3768204274 conn(0x7f0bd0077a20 msgr2=0x7f0bd01023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:20.824 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.207+0000 7f0bd83c2700 1 -- 172.21.15.120:0/3768204274 shutdown_connections 2023-06-23T01:11:20.824 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.207+0000 7f0bd83c2700 1 -- 172.21.15.120:0/3768204274 wait complete. 2023-06-23T01:11:20.825 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.207+0000 7f0bd83c2700 1 Processor -- start 2023-06-23T01:11:20.825 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.208+0000 7f0bd83c2700 1 -- start start 2023-06-23T01:11:20.825 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.208+0000 7f0bd83c2700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0bd0104470 0x7f0bd01130b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:20.825 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.208+0000 7f0bd83c2700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd01135f0 con 0x7f0bd0104470 2023-06-23T01:11:20.825 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.208+0000 7f0bd615e700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0bd0104470 0x7f0bd01130b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:20.825 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.208+0000 7f0bd615e700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0bd0104470 0x7f0bd01130b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42090/0 (socket says 172.21.15.120:42090) 2023-06-23T01:11:20.825 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.208+0000 7f0bd615e700 1 -- 172.21.15.120:0/1299157670 learned_addr learned my addr 172.21.15.120:0/1299157670 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:20.826 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.208+0000 7f0bd615e700 1 -- 172.21.15.120:0/1299157670 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bcc0094f0 con 0x7f0bd0104470 2023-06-23T01:11:20.826 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.209+0000 7f0bd615e700 1 --2- 172.21.15.120:0/1299157670 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0bd0104470 0x7f0bd01130b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00bce0 tx=0x7f0bcc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:20.826 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.209+0000 7f0bc6ffd700 1 -- 172.21.15.120:0/1299157670 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0bcc01a640 con 0x7f0bd0104470 2023-06-23T01:11:20.826 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.209+0000 7f0bc6ffd700 1 -- 172.21.15.120:0/1299157670 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0bcc01ac50 con 0x7f0bd0104470 2023-06-23T01:11:20.826 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.209+0000 7f0bd83c2700 1 -- 172.21.15.120:0/1299157670 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bd01137f0 con 0x7f0bd0104470 2023-06-23T01:11:20.826 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.209+0000 7f0bc6ffd700 1 -- 172.21.15.120:0/1299157670 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0bcc022b20 con 0x7f0bd0104470 2023-06-23T01:11:20.826 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.209+0000 7f0bd83c2700 1 -- 172.21.15.120:0/1299157670 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bd0114490 con 0x7f0bd0104470 2023-06-23T01:11:20.827 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.210+0000 7f0bc6ffd700 1 -- 172.21.15.120:0/1299157670 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 8) v1 ==== 43219+0+0 (secure 0 0 0) 0x7f0bcc022400 con 0x7f0bd0104470 2023-06-23T01:11:20.827 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.210+0000 7f0bd83c2700 1 -- 172.21.15.120:0/1299157670 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bd005f060 con 0x7f0bd0104470 2023-06-23T01:11:20.827 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.211+0000 7f0bc6ffd700 1 --2- 172.21.15.120:0/1299157670 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f0bbc0365c0 0x7f0bbc038a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:20.827 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.211+0000 7f0bc6ffd700 1 -- 172.21.15.120:0/1299157670 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f0bcc04a680 con 0x7f0bd0104470 2023-06-23T01:11:20.827 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.211+0000 7f0bd595d700 1 --2- 172.21.15.120:0/1299157670 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f0bbc0365c0 0x7f0bbc038a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:20.827 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.211+0000 7f0bd595d700 1 --2- 172.21.15.120:0/1299157670 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f0bbc0365c0 0x7f0bbc038a70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0009990 tx=0x7f0bc0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:20.827 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.215+0000 7f0bc6ffd700 1 -- 172.21.15.120:0/1299157670 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f0bcc02ab80 con 0x7f0bd0104470 2023-06-23T01:11:20.828 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.408+0000 7f0bd83c2700 1 -- 172.21.15.120:0/1299157670 --> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f0bd0102360 con 0x7f0bbc0365c0 2023-06-23T01:11:20.828 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.409+0000 7f0bc6ffd700 1 -- 172.21.15.120:0/1299157670 <== mgr.14120 v2:172.21.15.120:6800/1836713308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f0bd0102360 con 0x7f0bbc0365c0 2023-06-23T01:11:20.828 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.411+0000 7f0bd83c2700 1 -- 172.21.15.120:0/1299157670 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f0bbc0365c0 msgr2=0x7f0bbc038a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:20.828 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.411+0000 7f0bd83c2700 1 --2- 172.21.15.120:0/1299157670 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f0bbc0365c0 0x7f0bbc038a70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0009990 tx=0x7f0bc0006e40 comp rx=0 tx=0).stop 2023-06-23T01:11:20.828 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.411+0000 7f0bd83c2700 1 -- 172.21.15.120:0/1299157670 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0bd0104470 msgr2=0x7f0bd01130b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:20.828 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.411+0000 7f0bd83c2700 1 --2- 172.21.15.120:0/1299157670 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0bd0104470 0x7f0bd01130b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00bce0 tx=0x7f0bcc009250 comp rx=0 tx=0).stop 2023-06-23T01:11:20.828 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.411+0000 7f0bd83c2700 1 -- 172.21.15.120:0/1299157670 shutdown_connections 2023-06-23T01:11:20.829 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.411+0000 7f0bd83c2700 1 --2- 172.21.15.120:0/1299157670 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f0bbc0365c0 0x7f0bbc038a70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:20.829 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.411+0000 7f0bd83c2700 1 --2- 172.21.15.120:0/1299157670 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0bd0104470 0x7f0bd01130b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:20.829 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.411+0000 7f0bd83c2700 1 -- 172.21.15.120:0/1299157670 >> 172.21.15.120:0/1299157670 conn(0x7f0bd0077a20 msgr2=0x7f0bd0101c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:20.829 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.412+0000 7f0bd83c2700 1 -- 172.21.15.120:0/1299157670 shutdown_connections 2023-06-23T01:11:20.829 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:20.412+0000 7f0bd83c2700 1 -- 172.21.15.120:0/1299157670 wait complete. 2023-06-23T01:11:20.830 INFO:teuthology.orchestra.run.smithi120.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-06-23T01:11:20.830 INFO:teuthology.orchestra.run.smithi120.stdout:Adding key to root@localhost authorized_keys... 2023-06-23T01:11:20.831 INFO:teuthology.orchestra.run.smithi120.stdout:Adding host smithi120... 2023-06-23T01:11:21.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:21 smithi120 ceph-mon[79813]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:11:22.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:22 smithi120 ceph-mon[79813]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi120", "addr": "172.21.15.120", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:11:23.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:23 smithi120 ceph-mon[79813]: Deploying cephadm binary to smithi120 2023-06-23T01:11:24.503 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout Added host 'smithi120' with addr '172.21.15.120' 2023-06-23T01:11:24.504 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.316+0000 7f02da0c9700 1 Processor -- start 2023-06-23T01:11:24.504 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.316+0000 7f02da0c9700 1 -- start start 2023-06-23T01:11:24.504 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.316+0000 7f02da0c9700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02d4102210 0x7f02d4102620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:24.505 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.316+0000 7f02da0c9700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d4102b60 con 0x7f02d4102210 2023-06-23T01:11:24.505 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.316+0000 7f02d37fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02d4102210 0x7f02d4102620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:24.505 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.316+0000 7f02d37fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02d4102210 0x7f02d4102620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42096/0 (socket says 172.21.15.120:42096) 2023-06-23T01:11:24.505 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.316+0000 7f02d37fe700 1 -- 172.21.15.120:0/503537241 learned_addr learned my addr 172.21.15.120:0/503537241 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:24.506 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.317+0000 7f02d37fe700 1 -- 172.21.15.120:0/503537241 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d4102ca0 con 0x7f02d4102210 2023-06-23T01:11:24.506 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.317+0000 7f02d37fe700 1 --2- 172.21.15.120:0/503537241 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02d4102210 0x7f02d4102620 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f02c40098e0 tx=0x7f02c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88445209d9bcea90 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:24.506 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.317+0000 7f02d27fc700 1 -- 172.21.15.120:0/503537241 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f02c4009e80 con 0x7f02d4102210 2023-06-23T01:11:24.507 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.317+0000 7f02d27fc700 1 -- 172.21.15.120:0/503537241 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f02c400bd10 con 0x7f02d4102210 2023-06-23T01:11:24.507 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.317+0000 7f02d27fc700 1 -- 172.21.15.120:0/503537241 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f02c4011ce0 con 0x7f02d4102210 2023-06-23T01:11:24.507 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.318+0000 7f02da0c9700 1 -- 172.21.15.120:0/503537241 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02d4102210 msgr2=0x7f02d4102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:24.507 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.318+0000 7f02da0c9700 1 --2- 172.21.15.120:0/503537241 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02d4102210 0x7f02d4102620 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f02c40098e0 tx=0x7f02c4009250 comp rx=0 tx=0).stop 2023-06-23T01:11:24.507 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.318+0000 7f02da0c9700 1 -- 172.21.15.120:0/503537241 shutdown_connections 2023-06-23T01:11:24.507 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.318+0000 7f02da0c9700 1 --2- 172.21.15.120:0/503537241 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02d4102210 0x7f02d4102620 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:24.508 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.318+0000 7f02da0c9700 1 -- 172.21.15.120:0/503537241 >> 172.21.15.120:0/503537241 conn(0x7f02d40fd990 msgr2=0x7f02d40ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:24.508 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.318+0000 7f02da0c9700 1 -- 172.21.15.120:0/503537241 shutdown_connections 2023-06-23T01:11:24.508 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.318+0000 7f02da0c9700 1 -- 172.21.15.120:0/503537241 wait complete. 2023-06-23T01:11:24.508 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.319+0000 7f02da0c9700 1 Processor -- start 2023-06-23T01:11:24.508 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.319+0000 7f02da0c9700 1 -- start start 2023-06-23T01:11:24.508 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.319+0000 7f02da0c9700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02d4102210 0x7f02d4110eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:24.508 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.319+0000 7f02da0c9700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d41113f0 con 0x7f02d4102210 2023-06-23T01:11:24.509 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.319+0000 7f02d37fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02d4102210 0x7f02d4110eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:24.509 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.319+0000 7f02d37fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02d4102210 0x7f02d4110eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42098/0 (socket says 172.21.15.120:42098) 2023-06-23T01:11:24.509 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.319+0000 7f02d37fe700 1 -- 172.21.15.120:0/3054914021 learned_addr learned my addr 172.21.15.120:0/3054914021 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:24.509 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.320+0000 7f02d37fe700 1 -- 172.21.15.120:0/3054914021 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c40094f0 con 0x7f02d4102210 2023-06-23T01:11:24.509 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.320+0000 7f02d37fe700 1 --2- 172.21.15.120:0/3054914021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02d4102210 0x7f02d4110eb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f02c4006b00 tx=0x7f02c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:24.510 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.320+0000 7f02bbfff700 1 -- 172.21.15.120:0/3054914021 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f02c401a640 con 0x7f02d4102210 2023-06-23T01:11:24.510 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.320+0000 7f02bbfff700 1 -- 172.21.15.120:0/3054914021 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f02c401ac50 con 0x7f02d4102210 2023-06-23T01:11:24.510 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.320+0000 7f02da0c9700 1 -- 172.21.15.120:0/3054914021 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02d41115f0 con 0x7f02d4102210 2023-06-23T01:11:24.510 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.320+0000 7f02bbfff700 1 -- 172.21.15.120:0/3054914021 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f02c4022bb0 con 0x7f02d4102210 2023-06-23T01:11:24.510 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.320+0000 7f02da0c9700 1 -- 172.21.15.120:0/3054914021 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02d4112290 con 0x7f02d4102210 2023-06-23T01:11:24.510 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.321+0000 7f02bbfff700 1 -- 172.21.15.120:0/3054914021 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 8) v1 ==== 43219+0+0 (secure 0 0 0) 0x7f02c4022490 con 0x7f02d4102210 2023-06-23T01:11:24.511 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.321+0000 7f02da0c9700 1 -- 172.21.15.120:0/3054914021 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02d405f060 con 0x7f02d4102210 2023-06-23T01:11:24.511 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.322+0000 7f02bbfff700 1 --2- 172.21.15.120:0/3054914021 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f02bc0365c0 0x7f02bc038a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:24.511 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.322+0000 7f02bbfff700 1 -- 172.21.15.120:0/3054914021 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f02c404a480 con 0x7f02d4102210 2023-06-23T01:11:24.511 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.322+0000 7f02d2ffd700 1 --2- 172.21.15.120:0/3054914021 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f02bc0365c0 0x7f02bc038a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:24.511 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.323+0000 7f02d2ffd700 1 --2- 172.21.15.120:0/3054914021 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f02bc0365c0 0x7f02bc038a70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f02c8009990 tx=0x7f02c8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:24.511 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.327+0000 7f02bbfff700 1 -- 172.21.15.120:0/3054914021 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f02c401adc0 con 0x7f02d4102210 2023-06-23T01:11:24.511 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:21.522+0000 7f02da0c9700 1 -- 172.21.15.120:0/3054914021 --> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi120", "addr": "172.21.15.120", "target": ["mon-mgr", ""]}) v1 -- 0x7f02d40fecf0 con 0x7f02bc0365c0 2023-06-23T01:11:24.512 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:24.104+0000 7f02bbfff700 1 -- 172.21.15.120:0/3054914021 <== mgr.14120 v2:172.21.15.120:6800/1836713308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f02d40fecf0 con 0x7f02bc0365c0 2023-06-23T01:11:24.512 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:24.107+0000 7f02da0c9700 1 -- 172.21.15.120:0/3054914021 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f02bc0365c0 msgr2=0x7f02bc038a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:24.512 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:24.107+0000 7f02da0c9700 1 --2- 172.21.15.120:0/3054914021 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f02bc0365c0 0x7f02bc038a70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f02c8009990 tx=0x7f02c8006e40 comp rx=0 tx=0).stop 2023-06-23T01:11:24.512 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:24.107+0000 7f02da0c9700 1 -- 172.21.15.120:0/3054914021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02d4102210 msgr2=0x7f02d4110eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:24.512 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:24.107+0000 7f02da0c9700 1 --2- 172.21.15.120:0/3054914021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02d4102210 0x7f02d4110eb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f02c4006b00 tx=0x7f02c4009250 comp rx=0 tx=0).stop 2023-06-23T01:11:24.512 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:24.107+0000 7f02da0c9700 1 -- 172.21.15.120:0/3054914021 shutdown_connections 2023-06-23T01:11:24.513 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:24.107+0000 7f02da0c9700 1 --2- 172.21.15.120:0/3054914021 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f02bc0365c0 0x7f02bc038a70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:24.513 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:24.107+0000 7f02da0c9700 1 --2- 172.21.15.120:0/3054914021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02d4102210 0x7f02d4110eb0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:24.513 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:24.107+0000 7f02da0c9700 1 -- 172.21.15.120:0/3054914021 >> 172.21.15.120:0/3054914021 conn(0x7f02d40fd990 msgr2=0x7f02d40fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:24.513 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:24.107+0000 7f02da0c9700 1 -- 172.21.15.120:0/3054914021 shutdown_connections 2023-06-23T01:11:24.513 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:24.107+0000 7f02da0c9700 1 -- 172.21.15.120:0/3054914021 wait complete. 2023-06-23T01:11:24.513 INFO:teuthology.orchestra.run.smithi120.stdout:Deploying unmanaged mon service... 2023-06-23T01:11:25.415 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:25 smithi120 ceph-mon[79813]: from='mgr.14120 172.21.15.120:0/66681182' entity='mgr.x' 2023-06-23T01:11:25.415 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:25 smithi120 ceph-mon[79813]: Added host smithi120 2023-06-23T01:11:25.415 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:25 smithi120 ceph-mon[79813]: from='mgr.14120 172.21.15.120:0/66681182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:11:25.577 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-06-23T01:11:25.578 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.027+0000 7fd869ffb700 1 Processor -- start 2023-06-23T01:11:25.578 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.028+0000 7fd869ffb700 1 -- start start 2023-06-23T01:11:25.578 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.028+0000 7fd869ffb700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd864077cf0 0x7fd864076190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:25.578 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.028+0000 7fd869ffb700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8640766d0 con 0x7fd864077cf0 2023-06-23T01:11:25.578 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.028+0000 7fd8637fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd864077cf0 0x7fd864076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:25.578 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.028+0000 7fd8637fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd864077cf0 0x7fd864076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33654/0 (socket says 172.21.15.120:33654) 2023-06-23T01:11:25.579 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.028+0000 7fd8637fe700 1 -- 172.21.15.120:0/389331991 learned_addr learned my addr 172.21.15.120:0/389331991 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:25.579 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.029+0000 7fd8637fe700 1 -- 172.21.15.120:0/389331991 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd864076810 con 0x7fd864077cf0 2023-06-23T01:11:25.579 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.029+0000 7fd8637fe700 1 --2- 172.21.15.120:0/389331991 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd864077cf0 0x7fd864076190 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd8540098e0 tx=0x7fd854009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad91597f8cd2dc95 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:25.579 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.029+0000 7fd8627fc700 1 -- 172.21.15.120:0/389331991 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd854009e80 con 0x7fd864077cf0 2023-06-23T01:11:25.579 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.029+0000 7fd8627fc700 1 -- 172.21.15.120:0/389331991 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd85400bd10 con 0x7fd864077cf0 2023-06-23T01:11:25.579 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.029+0000 7fd8627fc700 1 -- 172.21.15.120:0/389331991 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd854011ce0 con 0x7fd864077cf0 2023-06-23T01:11:25.579 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.030+0000 7fd869ffb700 1 -- 172.21.15.120:0/389331991 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd864077cf0 msgr2=0x7fd864076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:25.580 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.030+0000 7fd869ffb700 1 --2- 172.21.15.120:0/389331991 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd864077cf0 0x7fd864076190 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd8540098e0 tx=0x7fd854009250 comp rx=0 tx=0).stop 2023-06-23T01:11:25.580 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.030+0000 7fd869ffb700 1 -- 172.21.15.120:0/389331991 shutdown_connections 2023-06-23T01:11:25.580 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.030+0000 7fd869ffb700 1 --2- 172.21.15.120:0/389331991 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd864077cf0 0x7fd864076190 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:25.580 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.030+0000 7fd869ffb700 1 -- 172.21.15.120:0/389331991 >> 172.21.15.120:0/389331991 conn(0x7fd8640fd990 msgr2=0x7fd8640ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:25.581 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.030+0000 7fd869ffb700 1 -- 172.21.15.120:0/389331991 shutdown_connections 2023-06-23T01:11:25.581 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.030+0000 7fd869ffb700 1 -- 172.21.15.120:0/389331991 wait complete. 2023-06-23T01:11:25.581 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.031+0000 7fd869ffb700 1 Processor -- start 2023-06-23T01:11:25.581 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.031+0000 7fd869ffb700 1 -- start start 2023-06-23T01:11:25.581 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.031+0000 7fd869ffb700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd864077cf0 0x7fd864113140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:25.581 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.031+0000 7fd869ffb700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd864113680 con 0x7fd864077cf0 2023-06-23T01:11:25.581 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.031+0000 7fd8637fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd864077cf0 0x7fd864113140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:25.582 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.031+0000 7fd8637fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd864077cf0 0x7fd864113140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33666/0 (socket says 172.21.15.120:33666) 2023-06-23T01:11:25.582 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.031+0000 7fd8637fe700 1 -- 172.21.15.120:0/3602445889 learned_addr learned my addr 172.21.15.120:0/3602445889 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:25.582 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.032+0000 7fd8637fe700 1 -- 172.21.15.120:0/3602445889 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8540094f0 con 0x7fd864077cf0 2023-06-23T01:11:25.582 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.032+0000 7fd8637fe700 1 --2- 172.21.15.120:0/3602445889 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd864077cf0 0x7fd864113140 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd854011970 tx=0x7fd854009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:25.582 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.032+0000 7fd84bfff700 1 -- 172.21.15.120:0/3602445889 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd854009e80 con 0x7fd864077cf0 2023-06-23T01:11:25.582 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.032+0000 7fd84bfff700 1 -- 172.21.15.120:0/3602445889 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd85401aaf0 con 0x7fd864077cf0 2023-06-23T01:11:25.582 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.032+0000 7fd869ffb700 1 -- 172.21.15.120:0/3602445889 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd864113880 con 0x7fd864077cf0 2023-06-23T01:11:25.583 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.032+0000 7fd84bfff700 1 -- 172.21.15.120:0/3602445889 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd854022a90 con 0x7fd864077cf0 2023-06-23T01:11:25.583 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.032+0000 7fd869ffb700 1 -- 172.21.15.120:0/3602445889 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd864114520 con 0x7fd864077cf0 2023-06-23T01:11:25.583 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.033+0000 7fd84bfff700 1 -- 172.21.15.120:0/3602445889 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 8) v1 ==== 43219+0+0 (secure 0 0 0) 0x7fd854022400 con 0x7fd864077cf0 2023-06-23T01:11:25.583 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.033+0000 7fd869ffb700 1 -- 172.21.15.120:0/3602445889 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd86405f060 con 0x7fd864077cf0 2023-06-23T01:11:25.583 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.034+0000 7fd84bfff700 1 --2- 172.21.15.120:0/3602445889 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7fd84c0365c0 0x7fd84c038a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:25.583 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.034+0000 7fd84bfff700 1 -- 172.21.15.120:0/3602445889 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fd85404a440 con 0x7fd864077cf0 2023-06-23T01:11:25.583 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.034+0000 7fd862ffd700 1 --2- 172.21.15.120:0/3602445889 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7fd84c0365c0 0x7fd84c038a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:25.583 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.035+0000 7fd862ffd700 1 --2- 172.21.15.120:0/3602445889 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7fd84c0365c0 0x7fd84c038a70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd858009990 tx=0x7fd858006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:25.584 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.039+0000 7fd84bfff700 1 -- 172.21.15.120:0/3602445889 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fd854006d70 con 0x7fd864077cf0 2023-06-23T01:11:25.584 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.255+0000 7fd869ffb700 1 -- 172.21.15.120:0/3602445889 --> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fd8640feda0 con 0x7fd84c0365c0 2023-06-23T01:11:25.584 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.261+0000 7fd84bfff700 1 -- 172.21.15.120:0/3602445889 <== mgr.14120 v2:172.21.15.120:6800/1836713308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fd8640feda0 con 0x7fd84c0365c0 2023-06-23T01:11:25.584 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.263+0000 7fd869ffb700 1 -- 172.21.15.120:0/3602445889 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7fd84c0365c0 msgr2=0x7fd84c038a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:25.584 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.263+0000 7fd869ffb700 1 --2- 172.21.15.120:0/3602445889 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7fd84c0365c0 0x7fd84c038a70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd858009990 tx=0x7fd858006e40 comp rx=0 tx=0).stop 2023-06-23T01:11:25.584 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.263+0000 7fd869ffb700 1 -- 172.21.15.120:0/3602445889 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd864077cf0 msgr2=0x7fd864113140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:25.584 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.263+0000 7fd869ffb700 1 --2- 172.21.15.120:0/3602445889 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd864077cf0 0x7fd864113140 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd854011970 tx=0x7fd854009250 comp rx=0 tx=0).stop 2023-06-23T01:11:25.585 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.263+0000 7fd869ffb700 1 -- 172.21.15.120:0/3602445889 shutdown_connections 2023-06-23T01:11:25.585 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.263+0000 7fd869ffb700 1 --2- 172.21.15.120:0/3602445889 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7fd84c0365c0 0x7fd84c038a70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:25.585 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.263+0000 7fd869ffb700 1 --2- 172.21.15.120:0/3602445889 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd864077cf0 0x7fd864113140 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:25.585 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.263+0000 7fd869ffb700 1 -- 172.21.15.120:0/3602445889 >> 172.21.15.120:0/3602445889 conn(0x7fd8640fd990 msgr2=0x7fd8640fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:25.585 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.263+0000 7fd869ffb700 1 -- 172.21.15.120:0/3602445889 shutdown_connections 2023-06-23T01:11:25.585 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:25.263+0000 7fd869ffb700 1 -- 172.21.15.120:0/3602445889 wait complete. 2023-06-23T01:11:25.585 INFO:teuthology.orchestra.run.smithi120.stdout:Deploying unmanaged mgr service... 2023-06-23T01:11:26.415 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:26 smithi120 ceph-mon[79813]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:11:26.415 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:26 smithi120 ceph-mon[79813]: Saving service mon spec with placement count:5 2023-06-23T01:11:26.415 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:26 smithi120 ceph-mon[79813]: from='mgr.14120 172.21.15.120:0/66681182' entity='mgr.x' 2023-06-23T01:11:26.854 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-06-23T01:11:26.854 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.187+0000 7f8af6f6d700 1 Processor -- start 2023-06-23T01:11:26.854 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.187+0000 7f8af6f6d700 1 -- start start 2023-06-23T01:11:26.855 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.188+0000 7f8af6f6d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8af0077ca0 0x7f8af00761b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:26.855 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.188+0000 7f8af6f6d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af00780b0 con 0x7f8af0077ca0 2023-06-23T01:11:26.855 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.188+0000 7f8af4d09700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8af0077ca0 0x7f8af00761b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:26.855 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.188+0000 7f8af4d09700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8af0077ca0 0x7f8af00761b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33674/0 (socket says 172.21.15.120:33674) 2023-06-23T01:11:26.855 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.188+0000 7f8af4d09700 1 -- 172.21.15.120:0/631388455 learned_addr learned my addr 172.21.15.120:0/631388455 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:26.855 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.189+0000 7f8af4d09700 1 -- 172.21.15.120:0/631388455 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af00766f0 con 0x7f8af0077ca0 2023-06-23T01:11:26.855 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.189+0000 7f8af4d09700 1 --2- 172.21.15.120:0/631388455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8af0077ca0 0x7f8af00761b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8ae40098e0 tx=0x7f8ae4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ecccaf7b310fb87 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:26.856 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.189+0000 7f8aef7fe700 1 -- 172.21.15.120:0/631388455 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ae4009e80 con 0x7f8af0077ca0 2023-06-23T01:11:26.856 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.189+0000 7f8aef7fe700 1 -- 172.21.15.120:0/631388455 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8ae400bd10 con 0x7f8af0077ca0 2023-06-23T01:11:26.856 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.189+0000 7f8aef7fe700 1 -- 172.21.15.120:0/631388455 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ae4011e20 con 0x7f8af0077ca0 2023-06-23T01:11:26.856 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.190+0000 7f8af6f6d700 1 -- 172.21.15.120:0/631388455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8af0077ca0 msgr2=0x7f8af00761b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:26.856 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.190+0000 7f8af6f6d700 1 --2- 172.21.15.120:0/631388455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8af0077ca0 0x7f8af00761b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8ae40098e0 tx=0x7f8ae4009250 comp rx=0 tx=0).stop 2023-06-23T01:11:26.856 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.190+0000 7f8af6f6d700 1 -- 172.21.15.120:0/631388455 shutdown_connections 2023-06-23T01:11:26.857 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.190+0000 7f8af6f6d700 1 --2- 172.21.15.120:0/631388455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8af0077ca0 0x7f8af00761b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:26.857 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.190+0000 7f8af6f6d700 1 -- 172.21.15.120:0/631388455 >> 172.21.15.120:0/631388455 conn(0x7f8af00fd9f0 msgr2=0x7f8af00ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:26.858 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.190+0000 7f8af6f6d700 1 -- 172.21.15.120:0/631388455 shutdown_connections 2023-06-23T01:11:26.858 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.190+0000 7f8af6f6d700 1 -- 172.21.15.120:0/631388455 wait complete. 2023-06-23T01:11:26.858 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.190+0000 7f8af6f6d700 1 Processor -- start 2023-06-23T01:11:26.858 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.191+0000 7f8af6f6d700 1 -- start start 2023-06-23T01:11:26.858 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.191+0000 7f8af6f6d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8af0077ca0 0x7f8af0113230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:26.859 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.191+0000 7f8af6f6d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af0113770 con 0x7f8af0077ca0 2023-06-23T01:11:26.859 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.191+0000 7f8af4d09700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8af0077ca0 0x7f8af0113230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:26.859 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.191+0000 7f8af4d09700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8af0077ca0 0x7f8af0113230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33680/0 (socket says 172.21.15.120:33680) 2023-06-23T01:11:26.859 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.191+0000 7f8af4d09700 1 -- 172.21.15.120:0/4248396190 learned_addr learned my addr 172.21.15.120:0/4248396190 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:26.859 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.192+0000 7f8af4d09700 1 -- 172.21.15.120:0/4248396190 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae40094f0 con 0x7f8af0077ca0 2023-06-23T01:11:26.860 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.192+0000 7f8af4d09700 1 --2- 172.21.15.120:0/4248396190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8af0077ca0 0x7f8af0113230 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4010640 tx=0x7f8ae4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:26.860 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.192+0000 7f8aed7fa700 1 -- 172.21.15.120:0/4248396190 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ae40117a0 con 0x7f8af0077ca0 2023-06-23T01:11:26.861 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.192+0000 7f8aed7fa700 1 -- 172.21.15.120:0/4248396190 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8ae4009e80 con 0x7f8af0077ca0 2023-06-23T01:11:26.861 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.192+0000 7f8af6f6d700 1 -- 172.21.15.120:0/4248396190 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af0113970 con 0x7f8af0077ca0 2023-06-23T01:11:26.861 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.192+0000 7f8aed7fa700 1 -- 172.21.15.120:0/4248396190 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ae401add0 con 0x7f8af0077ca0 2023-06-23T01:11:26.862 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.192+0000 7f8af6f6d700 1 -- 172.21.15.120:0/4248396190 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af0114610 con 0x7f8af0077ca0 2023-06-23T01:11:26.862 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.193+0000 7f8aed7fa700 1 -- 172.21.15.120:0/4248396190 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 8) v1 ==== 43219+0+0 (secure 0 0 0) 0x7f8ae401a430 con 0x7f8af0077ca0 2023-06-23T01:11:26.862 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.193+0000 7f8af6f6d700 1 -- 172.21.15.120:0/4248396190 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af005f060 con 0x7f8af0077ca0 2023-06-23T01:11:26.862 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.194+0000 7f8aed7fa700 1 --2- 172.21.15.120:0/4248396190 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f8ae00365c0 0x7f8ae0038a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:26.863 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.194+0000 7f8aed7fa700 1 -- 172.21.15.120:0/4248396190 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f8ae404a820 con 0x7f8af0077ca0 2023-06-23T01:11:26.863 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.194+0000 7f8aeffff700 1 --2- 172.21.15.120:0/4248396190 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f8ae00365c0 0x7f8ae0038a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:26.863 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.195+0000 7f8aeffff700 1 --2- 172.21.15.120:0/4248396190 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f8ae00365c0 0x7f8ae0038a70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8adc009990 tx=0x7f8adc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:26.864 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.198+0000 7f8aed7fa700 1 -- 172.21.15.120:0/4248396190 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f8ae401a710 con 0x7f8af0077ca0 2023-06-23T01:11:26.864 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.416+0000 7f8af6f6d700 1 -- 172.21.15.120:0/4248396190 --> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f8af00ffd20 con 0x7f8ae00365c0 2023-06-23T01:11:26.864 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.421+0000 7f8aed7fa700 1 -- 172.21.15.120:0/4248396190 <== mgr.14120 v2:172.21.15.120:6800/1836713308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f8af00ffd20 con 0x7f8ae00365c0 2023-06-23T01:11:26.864 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.423+0000 7f8af6f6d700 1 -- 172.21.15.120:0/4248396190 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f8ae00365c0 msgr2=0x7f8ae0038a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:26.865 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.423+0000 7f8af6f6d700 1 --2- 172.21.15.120:0/4248396190 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f8ae00365c0 0x7f8ae0038a70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8adc009990 tx=0x7f8adc006e40 comp rx=0 tx=0).stop 2023-06-23T01:11:26.865 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.423+0000 7f8af6f6d700 1 -- 172.21.15.120:0/4248396190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8af0077ca0 msgr2=0x7f8af0113230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:26.865 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.423+0000 7f8af6f6d700 1 --2- 172.21.15.120:0/4248396190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8af0077ca0 0x7f8af0113230 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4010640 tx=0x7f8ae4009250 comp rx=0 tx=0).stop 2023-06-23T01:11:26.866 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.423+0000 7f8af6f6d700 1 -- 172.21.15.120:0/4248396190 shutdown_connections 2023-06-23T01:11:26.866 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.423+0000 7f8af6f6d700 1 --2- 172.21.15.120:0/4248396190 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f8ae00365c0 0x7f8ae0038a70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:26.866 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.423+0000 7f8af6f6d700 1 --2- 172.21.15.120:0/4248396190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8af0077ca0 0x7f8af0113230 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:26.866 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.423+0000 7f8af6f6d700 1 -- 172.21.15.120:0/4248396190 >> 172.21.15.120:0/4248396190 conn(0x7f8af00fd9f0 msgr2=0x7f8af00ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:26.867 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.424+0000 7f8af6f6d700 1 -- 172.21.15.120:0/4248396190 shutdown_connections 2023-06-23T01:11:26.867 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:26.424+0000 7f8af6f6d700 1 -- 172.21.15.120:0/4248396190 wait complete. 2023-06-23T01:11:27.760 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:27 smithi120 ceph-mon[79813]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:11:27.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:27 smithi120 ceph-mon[79813]: Saving service mgr spec with placement count:2 2023-06-23T01:11:27.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:27 smithi120 ceph-mon[79813]: from='mgr.14120 172.21.15.120:0/66681182' entity='mgr.x' 2023-06-23T01:11:27.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:27 smithi120 ceph-mon[79813]: from='mgr.14120 172.21.15.120:0/66681182' entity='mgr.x' 2023-06-23T01:11:27.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:27 smithi120 ceph-mon[79813]: from='mgr.14120 172.21.15.120:0/66681182' entity='mgr.x' 2023-06-23T01:11:28.028 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.353+0000 7f830efea700 1 Processor -- start 2023-06-23T01:11:28.029 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.353+0000 7f830efea700 1 -- start start 2023-06-23T01:11:28.029 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.353+0000 7f830efea700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8308077cb0 0x7f8308076150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:28.029 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.353+0000 7f830efea700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83080780c0 con 0x7f8308077cb0 2023-06-23T01:11:28.030 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.354+0000 7f830cd86700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8308077cb0 0x7f8308076150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:28.030 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.354+0000 7f830cd86700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8308077cb0 0x7f8308076150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33694/0 (socket says 172.21.15.120:33694) 2023-06-23T01:11:28.030 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.354+0000 7f830cd86700 1 -- 172.21.15.120:0/4110179349 learned_addr learned my addr 172.21.15.120:0/4110179349 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:28.030 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.354+0000 7f830cd86700 1 -- 172.21.15.120:0/4110179349 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8308076690 con 0x7f8308077cb0 2023-06-23T01:11:28.031 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.354+0000 7f830cd86700 1 --2- 172.21.15.120:0/4110179349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8308077cb0 0x7f8308076150 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f82fc0098e0 tx=0x7f82fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f2194686eaa794 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:28.031 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.355+0000 7f83077fe700 1 -- 172.21.15.120:0/4110179349 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f82fc009e80 con 0x7f8308077cb0 2023-06-23T01:11:28.031 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.355+0000 7f83077fe700 1 -- 172.21.15.120:0/4110179349 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82fc00bd30 con 0x7f8308077cb0 2023-06-23T01:11:28.032 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.355+0000 7f83077fe700 1 -- 172.21.15.120:0/4110179349 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f82fc011ca0 con 0x7f8308077cb0 2023-06-23T01:11:28.032 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.355+0000 7f830efea700 1 -- 172.21.15.120:0/4110179349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8308077cb0 msgr2=0x7f8308076150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:28.032 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.355+0000 7f830efea700 1 --2- 172.21.15.120:0/4110179349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8308077cb0 0x7f8308076150 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f82fc0098e0 tx=0x7f82fc009250 comp rx=0 tx=0).stop 2023-06-23T01:11:28.033 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.355+0000 7f830efea700 1 -- 172.21.15.120:0/4110179349 shutdown_connections 2023-06-23T01:11:28.033 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.355+0000 7f830efea700 1 --2- 172.21.15.120:0/4110179349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8308077cb0 0x7f8308076150 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:28.033 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.355+0000 7f830efea700 1 -- 172.21.15.120:0/4110179349 >> 172.21.15.120:0/4110179349 conn(0x7f83080fd990 msgr2=0x7f83080ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:28.033 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.356+0000 7f830efea700 1 -- 172.21.15.120:0/4110179349 shutdown_connections 2023-06-23T01:11:28.034 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.356+0000 7f830efea700 1 -- 172.21.15.120:0/4110179349 wait complete. 2023-06-23T01:11:28.034 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.356+0000 7f830efea700 1 Processor -- start 2023-06-23T01:11:28.035 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.356+0000 7f830efea700 1 -- start start 2023-06-23T01:11:28.035 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.357+0000 7f830efea700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8308077cb0 0x7f8308113140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:28.035 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.357+0000 7f830efea700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8308113680 con 0x7f8308077cb0 2023-06-23T01:11:28.036 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.357+0000 7f830cd86700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8308077cb0 0x7f8308113140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:28.036 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.357+0000 7f830cd86700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8308077cb0 0x7f8308113140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33706/0 (socket says 172.21.15.120:33706) 2023-06-23T01:11:28.036 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.357+0000 7f830cd86700 1 -- 172.21.15.120:0/457957457 learned_addr learned my addr 172.21.15.120:0/457957457 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:28.037 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.357+0000 7f830cd86700 1 -- 172.21.15.120:0/457957457 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82fc0094f0 con 0x7f8308077cb0 2023-06-23T01:11:28.037 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.358+0000 7f830cd86700 1 --2- 172.21.15.120:0/457957457 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8308077cb0 0x7f8308113140 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f82fc006fd0 tx=0x7f82fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:28.037 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.358+0000 7f83057fa700 1 -- 172.21.15.120:0/457957457 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f82fc01a710 con 0x7f8308077cb0 2023-06-23T01:11:28.037 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.358+0000 7f83057fa700 1 -- 172.21.15.120:0/457957457 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82fc01ad40 con 0x7f8308077cb0 2023-06-23T01:11:28.038 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.358+0000 7f830efea700 1 -- 172.21.15.120:0/457957457 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8308113880 con 0x7f8308077cb0 2023-06-23T01:11:28.038 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.358+0000 7f83057fa700 1 -- 172.21.15.120:0/457957457 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f82fc022c90 con 0x7f8308077cb0 2023-06-23T01:11:28.038 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.358+0000 7f830efea700 1 -- 172.21.15.120:0/457957457 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8308113c90 con 0x7f8308077cb0 2023-06-23T01:11:28.039 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.359+0000 7f830efea700 1 -- 172.21.15.120:0/457957457 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f830804b970 con 0x7f8308077cb0 2023-06-23T01:11:28.039 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.360+0000 7f83057fa700 1 -- 172.21.15.120:0/457957457 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 8) v1 ==== 43219+0+0 (secure 0 0 0) 0x7f82fc01a870 con 0x7f8308077cb0 2023-06-23T01:11:28.039 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.360+0000 7f83057fa700 1 --2- 172.21.15.120:0/457957457 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f82f8036560 0x7f82f8038a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:28.039 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.360+0000 7f83057fa700 1 -- 172.21.15.120:0/457957457 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f82fc04a660 con 0x7f8308077cb0 2023-06-23T01:11:28.040 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.360+0000 7f8307fff700 1 --2- 172.21.15.120:0/457957457 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f82f8036560 0x7f82f8038a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:28.040 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.361+0000 7f8307fff700 1 --2- 172.21.15.120:0/457957457 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f82f8036560 0x7f82f8038a10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f82f4009990 tx=0x7f82f4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:28.040 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.365+0000 7f83057fa700 1 -- 172.21.15.120:0/457957457 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f82fc019e20 con 0x7f8308077cb0 2023-06-23T01:11:28.041 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.567+0000 7f830efea700 1 -- 172.21.15.120:0/457957457 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f8308108610 con 0x7f8308077cb0 2023-06-23T01:11:28.041 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.571+0000 7f83057fa700 1 -- 172.21.15.120:0/457957457 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f82fc019e20 con 0x7f8308077cb0 2023-06-23T01:11:28.042 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.573+0000 7f830efea700 1 -- 172.21.15.120:0/457957457 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f82f8036560 msgr2=0x7f82f8038a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:28.042 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.573+0000 7f830efea700 1 --2- 172.21.15.120:0/457957457 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f82f8036560 0x7f82f8038a10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f82f4009990 tx=0x7f82f4006e40 comp rx=0 tx=0).stop 2023-06-23T01:11:28.042 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.573+0000 7f830efea700 1 -- 172.21.15.120:0/457957457 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8308077cb0 msgr2=0x7f8308113140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:28.042 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.573+0000 7f830efea700 1 --2- 172.21.15.120:0/457957457 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8308077cb0 0x7f8308113140 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f82fc006fd0 tx=0x7f82fc009250 comp rx=0 tx=0).stop 2023-06-23T01:11:28.043 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.573+0000 7f830efea700 1 -- 172.21.15.120:0/457957457 shutdown_connections 2023-06-23T01:11:28.043 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.573+0000 7f830efea700 1 --2- 172.21.15.120:0/457957457 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f82f8036560 0x7f82f8038a10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:28.043 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.573+0000 7f830efea700 1 --2- 172.21.15.120:0/457957457 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8308077cb0 0x7f8308113140 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:28.043 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.573+0000 7f830efea700 1 -- 172.21.15.120:0/457957457 >> 172.21.15.120:0/457957457 conn(0x7f83080fd990 msgr2=0x7f83080fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:28.044 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.574+0000 7f830efea700 1 -- 172.21.15.120:0/457957457 shutdown_connections 2023-06-23T01:11:28.044 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:27.574+0000 7f830efea700 1 -- 172.21.15.120:0/457957457 wait complete. 2023-06-23T01:11:28.760 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:28 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/457957457' entity='client.admin' 2023-06-23T01:11:29.398 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.573+0000 7fe4a5367700 1 Processor -- start 2023-06-23T01:11:29.402 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.573+0000 7fe4a5367700 1 -- start start 2023-06-23T01:11:29.402 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.573+0000 7fe4a5367700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4a0070df0 0x7fe4a0104ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:29.402 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.573+0000 7fe4a5367700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a0105400 con 0x7fe4a0070df0 2023-06-23T01:11:29.402 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.574+0000 7fe49effd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4a0070df0 0x7fe4a0104ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:29.402 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.574+0000 7fe49effd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4a0070df0 0x7fe4a0104ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33708/0 (socket says 172.21.15.120:33708) 2023-06-23T01:11:29.402 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.574+0000 7fe49effd700 1 -- 172.21.15.120:0/187363857 learned_addr learned my addr 172.21.15.120:0/187363857 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:29.403 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.574+0000 7fe49effd700 1 -- 172.21.15.120:0/187363857 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a0105540 con 0x7fe4a0070df0 2023-06-23T01:11:29.403 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.574+0000 7fe49effd700 1 --2- 172.21.15.120:0/187363857 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4a0070df0 0x7fe4a0104ec0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe4900098e0 tx=0x7fe490009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809ad235e654736c server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:29.403 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.574+0000 7fe49dffb700 1 -- 172.21.15.120:0/187363857 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe490009e80 con 0x7fe4a0070df0 2023-06-23T01:11:29.403 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.574+0000 7fe49dffb700 1 -- 172.21.15.120:0/187363857 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe49000bd30 con 0x7fe4a0070df0 2023-06-23T01:11:29.403 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.574+0000 7fe49dffb700 1 -- 172.21.15.120:0/187363857 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe490011ca0 con 0x7fe4a0070df0 2023-06-23T01:11:29.403 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.575+0000 7fe4a5367700 1 -- 172.21.15.120:0/187363857 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4a0070df0 msgr2=0x7fe4a0104ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:29.403 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.575+0000 7fe4a5367700 1 --2- 172.21.15.120:0/187363857 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4a0070df0 0x7fe4a0104ec0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe4900098e0 tx=0x7fe490009250 comp rx=0 tx=0).stop 2023-06-23T01:11:29.404 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.575+0000 7fe4a5367700 1 -- 172.21.15.120:0/187363857 shutdown_connections 2023-06-23T01:11:29.404 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.575+0000 7fe4a5367700 1 --2- 172.21.15.120:0/187363857 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4a0070df0 0x7fe4a0104ec0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:29.404 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.575+0000 7fe4a5367700 1 -- 172.21.15.120:0/187363857 >> 172.21.15.120:0/187363857 conn(0x7fe4a00fc8f0 msgr2=0x7fe4a00fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:29.404 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.575+0000 7fe4a5367700 1 -- 172.21.15.120:0/187363857 shutdown_connections 2023-06-23T01:11:29.404 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.575+0000 7fe4a5367700 1 -- 172.21.15.120:0/187363857 wait complete. 2023-06-23T01:11:29.404 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.576+0000 7fe4a5367700 1 Processor -- start 2023-06-23T01:11:29.405 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.576+0000 7fe4a5367700 1 -- start start 2023-06-23T01:11:29.405 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.576+0000 7fe4a5367700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4a0070df0 0x7fe4a0102940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:29.405 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.576+0000 7fe4a5367700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a0100dc0 con 0x7fe4a0070df0 2023-06-23T01:11:29.405 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.577+0000 7fe49effd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4a0070df0 0x7fe4a0102940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:29.405 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.577+0000 7fe49effd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4a0070df0 0x7fe4a0102940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33724/0 (socket says 172.21.15.120:33724) 2023-06-23T01:11:29.405 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.577+0000 7fe49effd700 1 -- 172.21.15.120:0/337383331 learned_addr learned my addr 172.21.15.120:0/337383331 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:29.405 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.577+0000 7fe49effd700 1 -- 172.21.15.120:0/337383331 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4900094f0 con 0x7fe4a0070df0 2023-06-23T01:11:29.406 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.577+0000 7fe49effd700 1 --2- 172.21.15.120:0/337383331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4a0070df0 0x7fe4a0102940 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe490006fd0 tx=0x7fe490009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:29.406 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.577+0000 7fe4877fe700 1 -- 172.21.15.120:0/337383331 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe49001a710 con 0x7fe4a0070df0 2023-06-23T01:11:29.406 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.578+0000 7fe4877fe700 1 -- 172.21.15.120:0/337383331 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe49001ad40 con 0x7fe4a0070df0 2023-06-23T01:11:29.406 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.578+0000 7fe4a5367700 1 -- 172.21.15.120:0/337383331 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4a0100fc0 con 0x7fe4a0070df0 2023-06-23T01:11:29.406 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.578+0000 7fe4877fe700 1 -- 172.21.15.120:0/337383331 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe490022c90 con 0x7fe4a0070df0 2023-06-23T01:11:29.406 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.578+0000 7fe4a5367700 1 -- 172.21.15.120:0/337383331 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4a01013d0 con 0x7fe4a0070df0 2023-06-23T01:11:29.406 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.579+0000 7fe4877fe700 1 -- 172.21.15.120:0/337383331 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 8) v1 ==== 43219+0+0 (secure 0 0 0) 0x7fe49001a870 con 0x7fe4a0070df0 2023-06-23T01:11:29.407 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.579+0000 7fe4a5367700 1 -- 172.21.15.120:0/337383331 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4a004b970 con 0x7fe4a0070df0 2023-06-23T01:11:29.407 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.579+0000 7fe4877fe700 1 --2- 172.21.15.120:0/337383331 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7fe488036560 0x7fe488038a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:29.407 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.579+0000 7fe4877fe700 1 -- 172.21.15.120:0/337383331 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fe49004a680 con 0x7fe4a0070df0 2023-06-23T01:11:29.408 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.579+0000 7fe49e7fc700 1 --2- 172.21.15.120:0/337383331 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7fe488036560 0x7fe488038a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:29.408 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.580+0000 7fe49e7fc700 1 --2- 172.21.15.120:0/337383331 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7fe488036560 0x7fe488038a10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe494009990 tx=0x7fe494006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:29.409 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.584+0000 7fe4877fe700 1 -- 172.21.15.120:0/337383331 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fe4900217c0 con 0x7fe4a0070df0 2023-06-23T01:11:29.409 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.792+0000 7fe4a5367700 1 -- 172.21.15.120:0/337383331 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fe4a0101910 con 0x7fe4a0070df0 2023-06-23T01:11:29.409 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.796+0000 7fe4877fe700 1 -- 172.21.15.120:0/337383331 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fe4900217c0 con 0x7fe4a0070df0 2023-06-23T01:11:29.409 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.799+0000 7fe4a5367700 1 -- 172.21.15.120:0/337383331 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7fe488036560 msgr2=0x7fe488038a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:29.409 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.799+0000 7fe4a5367700 1 --2- 172.21.15.120:0/337383331 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7fe488036560 0x7fe488038a10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe494009990 tx=0x7fe494006e40 comp rx=0 tx=0).stop 2023-06-23T01:11:29.409 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.799+0000 7fe4a5367700 1 -- 172.21.15.120:0/337383331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4a0070df0 msgr2=0x7fe4a0102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:29.409 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.799+0000 7fe4a5367700 1 --2- 172.21.15.120:0/337383331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4a0070df0 0x7fe4a0102940 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe490006fd0 tx=0x7fe490009250 comp rx=0 tx=0).stop 2023-06-23T01:11:29.410 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.799+0000 7fe4a5367700 1 -- 172.21.15.120:0/337383331 shutdown_connections 2023-06-23T01:11:29.410 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.799+0000 7fe4a5367700 1 --2- 172.21.15.120:0/337383331 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7fe488036560 0x7fe488038a10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:29.410 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.799+0000 7fe4a5367700 1 --2- 172.21.15.120:0/337383331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4a0070df0 0x7fe4a0102940 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:29.410 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.799+0000 7fe4a5367700 1 -- 172.21.15.120:0/337383331 >> 172.21.15.120:0/337383331 conn(0x7fe4a00fc8f0 msgr2=0x7fe4a00fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:29.410 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.799+0000 7fe4a5367700 1 -- 172.21.15.120:0/337383331 shutdown_connections 2023-06-23T01:11:29.410 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:28.800+0000 7fe4a5367700 1 -- 172.21.15.120:0/337383331 wait complete. 2023-06-23T01:11:29.410 INFO:teuthology.orchestra.run.smithi120.stdout:Enabling the dashboard module... 2023-06-23T01:11:29.980 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:29 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/337383331' entity='client.admin' 2023-06-23T01:11:29.980 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:29 smithi120 ceph-mon[79813]: from='mgr.14120 172.21.15.120:0/66681182' entity='mgr.x' 2023-06-23T01:11:31.183 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.939+0000 7fb3f81e0700 1 Processor -- start 2023-06-23T01:11:31.183 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.939+0000 7fb3f81e0700 1 -- start start 2023-06-23T01:11:31.183 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.940+0000 7fb3f81e0700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb3f0102270 0x7fb3f0102680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:31.183 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.940+0000 7fb3f81e0700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f0102bc0 con 0x7fb3f0102270 2023-06-23T01:11:31.183 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.940+0000 7fb3f5f7c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb3f0102270 0x7fb3f0102680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:31.183 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.940+0000 7fb3f5f7c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb3f0102270 0x7fb3f0102680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33734/0 (socket says 172.21.15.120:33734) 2023-06-23T01:11:31.184 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.940+0000 7fb3f5f7c700 1 -- 172.21.15.120:0/1731121662 learned_addr learned my addr 172.21.15.120:0/1731121662 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:31.184 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.940+0000 7fb3f5f7c700 1 -- 172.21.15.120:0/1731121662 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f0102d00 con 0x7fb3f0102270 2023-06-23T01:11:31.184 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.940+0000 7fb3f5f7c700 1 --2- 172.21.15.120:0/1731121662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb3f0102270 0x7fb3f0102680 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec0098e0 tx=0x7fb3ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5334efe3a4101e35 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:31.184 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.941+0000 7fb3f4f7a700 1 -- 172.21.15.120:0/1731121662 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3ec009e80 con 0x7fb3f0102270 2023-06-23T01:11:31.184 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.941+0000 7fb3f4f7a700 1 -- 172.21.15.120:0/1731121662 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3ec00bd30 con 0x7fb3f0102270 2023-06-23T01:11:31.185 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.941+0000 7fb3f4f7a700 1 -- 172.21.15.120:0/1731121662 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3ec011ca0 con 0x7fb3f0102270 2023-06-23T01:11:31.185 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.941+0000 7fb3f81e0700 1 -- 172.21.15.120:0/1731121662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb3f0102270 msgr2=0x7fb3f0102680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:31.185 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.941+0000 7fb3f81e0700 1 --2- 172.21.15.120:0/1731121662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb3f0102270 0x7fb3f0102680 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec0098e0 tx=0x7fb3ec009250 comp rx=0 tx=0).stop 2023-06-23T01:11:31.185 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.941+0000 7fb3f81e0700 1 -- 172.21.15.120:0/1731121662 shutdown_connections 2023-06-23T01:11:31.185 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.941+0000 7fb3f81e0700 1 --2- 172.21.15.120:0/1731121662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb3f0102270 0x7fb3f0102680 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:31.185 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.941+0000 7fb3f81e0700 1 -- 172.21.15.120:0/1731121662 >> 172.21.15.120:0/1731121662 conn(0x7fb3f00fd9f0 msgr2=0x7fb3f00ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:31.186 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.941+0000 7fb3f81e0700 1 -- 172.21.15.120:0/1731121662 shutdown_connections 2023-06-23T01:11:31.186 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.941+0000 7fb3f81e0700 1 -- 172.21.15.120:0/1731121662 wait complete. 2023-06-23T01:11:31.186 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.942+0000 7fb3f81e0700 1 Processor -- start 2023-06-23T01:11:31.186 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.942+0000 7fb3f81e0700 1 -- start start 2023-06-23T01:11:31.186 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.943+0000 7fb3f81e0700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb3f0102270 0x7fb3f0110ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:31.186 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.943+0000 7fb3f81e0700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f0111410 con 0x7fb3f0102270 2023-06-23T01:11:31.187 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.943+0000 7fb3f5f7c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb3f0102270 0x7fb3f0110ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:31.187 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.943+0000 7fb3f5f7c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb3f0102270 0x7fb3f0110ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33738/0 (socket says 172.21.15.120:33738) 2023-06-23T01:11:31.187 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.943+0000 7fb3f5f7c700 1 -- 172.21.15.120:0/1327699352 learned_addr learned my addr 172.21.15.120:0/1327699352 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:31.187 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.943+0000 7fb3f5f7c700 1 -- 172.21.15.120:0/1327699352 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3ec0094f0 con 0x7fb3f0102270 2023-06-23T01:11:31.187 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.943+0000 7fb3f5f7c700 1 --2- 172.21.15.120:0/1327699352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb3f0102270 0x7fb3f0110ed0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec006b00 tx=0x7fb3ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:31.187 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.944+0000 7fb3e67fc700 1 -- 172.21.15.120:0/1327699352 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3ec018660 con 0x7fb3f0102270 2023-06-23T01:11:31.187 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.944+0000 7fb3e67fc700 1 -- 172.21.15.120:0/1327699352 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3ec018c90 con 0x7fb3f0102270 2023-06-23T01:11:31.188 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.944+0000 7fb3f81e0700 1 -- 172.21.15.120:0/1327699352 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f0111610 con 0x7fb3f0102270 2023-06-23T01:11:31.188 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.944+0000 7fb3e67fc700 1 -- 172.21.15.120:0/1327699352 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3ec01abf0 con 0x7fb3f0102270 2023-06-23T01:11:31.188 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.944+0000 7fb3f81e0700 1 -- 172.21.15.120:0/1327699352 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f0111a20 con 0x7fb3f0102270 2023-06-23T01:11:31.188 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.945+0000 7fb3e67fc700 1 -- 172.21.15.120:0/1327699352 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 8) v1 ==== 43219+0+0 (secure 0 0 0) 0x7fb3ec01a530 con 0x7fb3f0102270 2023-06-23T01:11:31.188 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.945+0000 7fb3f81e0700 1 -- 172.21.15.120:0/1327699352 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f004b970 con 0x7fb3f0102270 2023-06-23T01:11:31.188 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.945+0000 7fb3e67fc700 1 --2- 172.21.15.120:0/1327699352 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7fb3dc0365c0 0x7fb3dc038a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:31.189 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.945+0000 7fb3e67fc700 1 -- 172.21.15.120:0/1327699352 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fb3ec04a6a0 con 0x7fb3f0102270 2023-06-23T01:11:31.189 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.946+0000 7fb3f577b700 1 --2- 172.21.15.120:0/1327699352 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7fb3dc0365c0 0x7fb3dc038a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:31.189 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.946+0000 7fb3f577b700 1 --2- 172.21.15.120:0/1327699352 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7fb3dc0365c0 0x7fb3dc038a70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0009990 tx=0x7fb3e0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:31.189 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:29.950+0000 7fb3e67fc700 1 -- 172.21.15.120:0/1327699352 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fb3ec020350 con 0x7fb3f0102270 2023-06-23T01:11:31.189 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:30.211+0000 7fb3f81e0700 1 -- 172.21.15.120:0/1327699352 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fb3f0112690 con 0x7fb3f0102270 2023-06-23T01:11:31.189 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:30.803+0000 7fb3e67fc700 1 -- 172.21.15.120:0/1327699352 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mgrmap(e 9) v1 ==== 43232+0+0 (secure 0 0 0) 0x7fb3ec02c960 con 0x7fb3f0102270 2023-06-23T01:11:31.189 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:30.805+0000 7fb3e67fc700 1 -- 172.21.15.120:0/1327699352 <== mon.0 v2:172.21.15.120:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fb3ec020580 con 0x7fb3f0102270 2023-06-23T01:11:31.189 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:30.807+0000 7fb3f81e0700 1 -- 172.21.15.120:0/1327699352 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7fb3dc0365c0 msgr2=0x7fb3dc038a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:31.190 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:30.807+0000 7fb3f81e0700 1 --2- 172.21.15.120:0/1327699352 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7fb3dc0365c0 0x7fb3dc038a70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0009990 tx=0x7fb3e0006e40 comp rx=0 tx=0).stop 2023-06-23T01:11:31.190 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:30.807+0000 7fb3f81e0700 1 -- 172.21.15.120:0/1327699352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb3f0102270 msgr2=0x7fb3f0110ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:31.190 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:30.807+0000 7fb3f81e0700 1 --2- 172.21.15.120:0/1327699352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb3f0102270 0x7fb3f0110ed0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec006b00 tx=0x7fb3ec009250 comp rx=0 tx=0).stop 2023-06-23T01:11:31.190 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:30.807+0000 7fb3f81e0700 1 -- 172.21.15.120:0/1327699352 shutdown_connections 2023-06-23T01:11:31.190 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:30.807+0000 7fb3f81e0700 1 --2- 172.21.15.120:0/1327699352 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7fb3dc0365c0 0x7fb3dc038a70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:31.190 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:30.807+0000 7fb3f81e0700 1 --2- 172.21.15.120:0/1327699352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb3f0102270 0x7fb3f0110ed0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:31.190 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:30.807+0000 7fb3f81e0700 1 -- 172.21.15.120:0/1327699352 >> 172.21.15.120:0/1327699352 conn(0x7fb3f00fd9f0 msgr2=0x7fb3f00fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:31.191 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:30.807+0000 7fb3f81e0700 1 -- 172.21.15.120:0/1327699352 shutdown_connections 2023-06-23T01:11:31.191 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:30.807+0000 7fb3f81e0700 1 -- 172.21.15.120:0/1327699352 wait complete. 2023-06-23T01:11:31.211 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:30 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/1327699352' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-06-23T01:11:31.212 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:30 smithi120 conmon[80224]: ignoring --setuser ceph since I am not root 2023-06-23T01:11:31.212 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:30 smithi120 conmon[80224]: ignoring --setgroup ceph since I am not root 2023-06-23T01:11:32.171 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:31 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/1327699352' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-06-23T01:11:32.172 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:31 smithi120 ceph-mon[79813]: mgrmap e9: x(active, since 16s) 2023-06-23T01:11:32.359 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout { 2023-06-23T01:11:32.360 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-06-23T01:11:32.360 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "available": true, 2023-06-23T01:11:32.360 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-06-23T01:11:32.360 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-06-23T01:11:32.360 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout } 2023-06-23T01:11:32.360 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.740+0000 7f6fc0a76700 1 Processor -- start 2023-06-23T01:11:32.361 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.740+0000 7f6fc0a76700 1 -- start start 2023-06-23T01:11:32.361 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.741+0000 7f6fc0a76700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6fbc077cb0 0x7f6fbc076150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:32.361 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.741+0000 7f6fc0a76700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc0780c0 con 0x7f6fbc077cb0 2023-06-23T01:11:32.361 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.741+0000 7f6fba59c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6fbc077cb0 0x7f6fbc076150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:32.361 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.741+0000 7f6fba59c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6fbc077cb0 0x7f6fbc076150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53166/0 (socket says 172.21.15.120:53166) 2023-06-23T01:11:32.361 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.741+0000 7f6fba59c700 1 -- 172.21.15.120:0/4067735569 learned_addr learned my addr 172.21.15.120:0/4067735569 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:32.361 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.742+0000 7f6fba59c700 1 -- 172.21.15.120:0/4067735569 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fbc076690 con 0x7f6fbc077cb0 2023-06-23T01:11:32.362 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.742+0000 7f6fba59c700 1 --2- 172.21.15.120:0/4067735569 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6fbc077cb0 0x7f6fbc076150 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6fac0098e0 tx=0x7f6fac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ad7572c540e8ee server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:32.362 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.742+0000 7f6fb959a700 1 -- 172.21.15.120:0/4067735569 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6fac009e80 con 0x7f6fbc077cb0 2023-06-23T01:11:32.362 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.742+0000 7f6fb959a700 1 -- 172.21.15.120:0/4067735569 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fac00bd30 con 0x7f6fbc077cb0 2023-06-23T01:11:32.362 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.742+0000 7f6fb959a700 1 -- 172.21.15.120:0/4067735569 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6fac011ca0 con 0x7f6fbc077cb0 2023-06-23T01:11:32.362 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.743+0000 7f6fc0a76700 1 -- 172.21.15.120:0/4067735569 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6fbc077cb0 msgr2=0x7f6fbc076150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:32.362 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.743+0000 7f6fc0a76700 1 --2- 172.21.15.120:0/4067735569 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6fbc077cb0 0x7f6fbc076150 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6fac0098e0 tx=0x7f6fac009250 comp rx=0 tx=0).stop 2023-06-23T01:11:32.362 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.743+0000 7f6fc0a76700 1 -- 172.21.15.120:0/4067735569 shutdown_connections 2023-06-23T01:11:32.362 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.743+0000 7f6fc0a76700 1 --2- 172.21.15.120:0/4067735569 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6fbc077cb0 0x7f6fbc076150 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:32.363 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.743+0000 7f6fc0a76700 1 -- 172.21.15.120:0/4067735569 >> 172.21.15.120:0/4067735569 conn(0x7f6fbc0fd990 msgr2=0x7f6fbc0ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:32.363 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.743+0000 7f6fc0a76700 1 -- 172.21.15.120:0/4067735569 shutdown_connections 2023-06-23T01:11:32.363 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.743+0000 7f6fc0a76700 1 -- 172.21.15.120:0/4067735569 wait complete. 2023-06-23T01:11:32.363 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.744+0000 7f6fc0a76700 1 Processor -- start 2023-06-23T01:11:32.363 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.744+0000 7f6fc0a76700 1 -- start start 2023-06-23T01:11:32.363 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.744+0000 7f6fc0a76700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6fbc077cb0 0x7f6fbc113110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:32.363 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.744+0000 7f6fc0a76700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc113650 con 0x7f6fbc077cb0 2023-06-23T01:11:32.363 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.745+0000 7f6fba59c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6fbc077cb0 0x7f6fbc113110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:32.364 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.745+0000 7f6fba59c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6fbc077cb0 0x7f6fbc113110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53168/0 (socket says 172.21.15.120:53168) 2023-06-23T01:11:32.364 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.745+0000 7f6fba59c700 1 -- 172.21.15.120:0/4191084598 learned_addr learned my addr 172.21.15.120:0/4191084598 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:32.365 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.745+0000 7f6fba59c700 1 -- 172.21.15.120:0/4191084598 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fac0094f0 con 0x7f6fbc077cb0 2023-06-23T01:11:32.365 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.745+0000 7f6fba59c700 1 --2- 172.21.15.120:0/4191084598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6fbc077cb0 0x7f6fbc113110 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6fac006fd0 tx=0x7f6fac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:32.365 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.746+0000 7f6faaffd700 1 -- 172.21.15.120:0/4191084598 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6fac01a710 con 0x7f6fbc077cb0 2023-06-23T01:11:32.365 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.746+0000 7f6fc0a76700 1 -- 172.21.15.120:0/4191084598 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fbc113850 con 0x7f6fbc077cb0 2023-06-23T01:11:32.365 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.746+0000 7f6faaffd700 1 -- 172.21.15.120:0/4191084598 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fac01ad40 con 0x7f6fbc077cb0 2023-06-23T01:11:32.365 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.746+0000 7f6faaffd700 1 -- 172.21.15.120:0/4191084598 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6fac022c90 con 0x7f6fbc077cb0 2023-06-23T01:11:32.366 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.746+0000 7f6fc0a76700 1 -- 172.21.15.120:0/4191084598 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fbc113c60 con 0x7f6fbc077cb0 2023-06-23T01:11:32.366 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.747+0000 7f6faaffd700 1 -- 172.21.15.120:0/4191084598 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 9) v1 ==== 43232+0+0 (secure 0 0 0) 0x7f6fac02c440 con 0x7f6fbc077cb0 2023-06-23T01:11:32.366 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.747+0000 7f6fc0a76700 1 -- 172.21.15.120:0/4191084598 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fbc04b970 con 0x7f6fbc077cb0 2023-06-23T01:11:32.366 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.747+0000 7f6faaffd700 1 --2- 172.21.15.120:0/4191084598 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f6fa40365c0 0x7f6fa4038a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:32.366 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.747+0000 7f6faaffd700 1 -- 172.21.15.120:0/4191084598 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f6fac04a7e0 con 0x7f6fbc077cb0 2023-06-23T01:11:32.366 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.747+0000 7f6fb9d9b700 1 -- 172.21.15.120:0/4191084598 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f6fa40365c0 msgr2=0x7f6fa4038a70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.120:6800/1836713308 2023-06-23T01:11:32.366 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.747+0000 7f6fb9d9b700 1 --2- 172.21.15.120:0/4191084598 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f6fa40365c0 0x7f6fa4038a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-23T01:11:32.367 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.752+0000 7f6faaffd700 1 -- 172.21.15.120:0/4191084598 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f6fac020350 con 0x7f6fbc077cb0 2023-06-23T01:11:32.367 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.948+0000 7f6fb9d9b700 1 -- 172.21.15.120:0/4191084598 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f6fa40365c0 msgr2=0x7f6fa4038a70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.120:6800/1836713308 2023-06-23T01:11:32.367 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:31.948+0000 7f6fb9d9b700 1 --2- 172.21.15.120:0/4191084598 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f6fa40365c0 0x7f6fa4038a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-06-23T01:11:32.367 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.004+0000 7f6fc0a76700 1 -- 172.21.15.120:0/4191084598 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f6fbc1148d0 con 0x7f6fbc077cb0 2023-06-23T01:11:32.367 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.005+0000 7f6faaffd700 1 -- 172.21.15.120:0/4191084598 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f6fac018b40 con 0x7f6fbc077cb0 2023-06-23T01:11:32.367 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.007+0000 7f6fc0a76700 1 -- 172.21.15.120:0/4191084598 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f6fa40365c0 msgr2=0x7f6fa4038a70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-23T01:11:32.367 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.007+0000 7f6fc0a76700 1 --2- 172.21.15.120:0/4191084598 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f6fa40365c0 0x7f6fa4038a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:32.368 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.007+0000 7f6fc0a76700 1 -- 172.21.15.120:0/4191084598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6fbc077cb0 msgr2=0x7f6fbc113110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:32.368 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.007+0000 7f6fc0a76700 1 --2- 172.21.15.120:0/4191084598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6fbc077cb0 0x7f6fbc113110 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6fac006fd0 tx=0x7f6fac009250 comp rx=0 tx=0).stop 2023-06-23T01:11:32.368 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.007+0000 7f6fc0a76700 1 -- 172.21.15.120:0/4191084598 shutdown_connections 2023-06-23T01:11:32.368 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.007+0000 7f6fc0a76700 1 --2- 172.21.15.120:0/4191084598 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f6fa40365c0 0x7f6fa4038a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:32.368 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.007+0000 7f6fc0a76700 1 --2- 172.21.15.120:0/4191084598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6fbc077cb0 0x7f6fbc113110 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:32.368 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.007+0000 7f6fc0a76700 1 -- 172.21.15.120:0/4191084598 >> 172.21.15.120:0/4191084598 conn(0x7f6fbc0fd990 msgr2=0x7f6fbc0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:32.369 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.008+0000 7f6fc0a76700 1 -- 172.21.15.120:0/4191084598 shutdown_connections 2023-06-23T01:11:32.369 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.008+0000 7f6fc0a76700 1 -- 172.21.15.120:0/4191084598 wait complete. 2023-06-23T01:11:32.369 INFO:teuthology.orchestra.run.smithi120.stdout:Waiting for the mgr to restart... 2023-06-23T01:11:32.369 INFO:teuthology.orchestra.run.smithi120.stdout:Waiting for mgr epoch 9... 2023-06-23T01:11:32.859 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:32 smithi120 conmon[80224]: 2023-06-23T01:11:32.516+0000 7f39242a41c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-06-23T01:11:32.859 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:32 smithi120 conmon[80224]: 2023-06-23T01:11:32.736+0000 7f39242a41c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-06-23T01:11:33.124 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:32 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/4191084598' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-06-23T01:11:33.124 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:32 smithi120 conmon[80224]: 2023-06-23T01:11:32.854+0000 7f39242a41c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-06-23T01:11:33.510 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:33 smithi120 conmon[80224]: 2023-06-23T01:11:33.120+0000 7f39242a41c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-06-23T01:11:33.510 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:33 smithi120 conmon[80224]: 2023-06-23T01:11:33.367+0000 7f39242a41c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-06-23T01:11:34.010 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:33 smithi120 conmon[80224]: 2023-06-23T01:11:33.556+0000 7f39242a41c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-06-23T01:11:34.010 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:33 smithi120 conmon[80224]: 2023-06-23T01:11:33.657+0000 7f39242a41c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-06-23T01:11:34.760 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:34 smithi120 conmon[80224]: 2023-06-23T01:11:34.397+0000 7f39242a41c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-06-23T01:11:35.260 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:34 smithi120 conmon[80224]: 2023-06-23T01:11:34.985+0000 7f39242a41c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-06-23T01:11:35.260 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:35 smithi120 conmon[80224]: 2023-06-23T01:11:35.100+0000 7f39242a41c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-06-23T01:11:35.261 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:35 smithi120 conmon[80224]: 2023-06-23T01:11:35.216+0000 7f39242a41c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-06-23T01:11:36.260 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:35 smithi120 conmon[80224]: 2023-06-23T01:11:35.958+0000 7f39242a41c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-06-23T01:11:36.260 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:36 smithi120 conmon[80224]: 2023-06-23T01:11:36.170+0000 7f39242a41c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-06-23T01:11:36.570 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:36 smithi120 conmon[80224]: 2023-06-23T01:11:36.286+0000 7f39242a41c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-06-23T01:11:36.570 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:36 smithi120 conmon[80224]: 2023-06-23T01:11:36.451+0000 7f39242a41c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-06-23T01:11:36.570 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:36 smithi120 conmon[80224]: 2023-06-23T01:11:36.567+0000 7f39242a41c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-06-23T01:11:38.010 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:37 smithi120 conmon[80224]: 2023-06-23T01:11:37.626+0000 7f39242a41c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-06-23T01:11:38.010 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:37 smithi120 conmon[80224]: 2023-06-23T01:11:37.736+0000 7f39242a41c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-06-23T01:11:38.010 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:37 smithi120 conmon[80224]: 2023-06-23T01:11:37.848+0000 7f39242a41c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-06-23T01:11:38.510 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:38 smithi120 conmon[80224]: 2023-06-23T01:11:38.033+0000 7f39242a41c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-06-23T01:11:38.510 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:38 smithi120 conmon[80224]: 2023-06-23T01:11:38.158+0000 7f39242a41c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-06-23T01:11:38.967 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:38 smithi120 conmon[80224]: 2023-06-23T01:11:38.626+0000 7f39242a41c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-06-23T01:11:38.968 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:38 smithi120 conmon[80224]: 2023-06-23T01:11:38.740+0000 7f39242a41c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-06-23T01:11:38.968 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:38 smithi120 conmon[80224]: 2023-06-23T01:11:38.852+0000 7f39242a41c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-06-23T01:11:38.968 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:11:38 smithi120 conmon[80224]: 2023-06-23T01:11:38.964+0000 7f39242a41c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-06-23T01:11:39.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:39 smithi120 ceph-mon[79813]: Active manager daemon x restarted 2023-06-23T01:11:39.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:39 smithi120 ceph-mon[79813]: Activating manager daemon x 2023-06-23T01:11:39.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:39 smithi120 ceph-mon[79813]: osdmap e3: 0 total, 0 up, 0 in 2023-06-23T01:11:39.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:39 smithi120 ceph-mon[79813]: mgrmap e10: x(active, starting, since 0.00391178s) 2023-06-23T01:11:39.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:39 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-23T01:11:39.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:39 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-06-23T01:11:39.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:39 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-06-23T01:11:39.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:39 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-06-23T01:11:39.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:39 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-06-23T01:11:39.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:39 smithi120 ceph-mon[79813]: Manager daemon x is now available 2023-06-23T01:11:39.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:39 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:11:39.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:39 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:11:40.367 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:40 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-06-23T01:11:40.367 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:40 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-06-23T01:11:40.367 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:40 smithi120 ceph-mon[79813]: [23/Jun/2023:01:11:39] ENGINE Bus STARTING 2023-06-23T01:11:40.367 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:40 smithi120 ceph-mon[79813]: [23/Jun/2023:01:11:39] ENGINE Serving on https://172.21.15.120:7150 2023-06-23T01:11:40.367 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:40 smithi120 ceph-mon[79813]: [23/Jun/2023:01:11:39] ENGINE Bus STARTED 2023-06-23T01:11:40.367 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:40 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:11:40.368 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:40 smithi120 ceph-mon[79813]: mgrmap e11: x(active, since 1.00926s) 2023-06-23T01:11:40.444 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout { 2023-06-23T01:11:40.444 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-06-23T01:11:40.444 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "initialized": true 2023-06-23T01:11:40.445 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout } 2023-06-23T01:11:40.445 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.900+0000 7f0b9a0cb700 1 Processor -- start 2023-06-23T01:11:40.445 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.900+0000 7f0b9a0cb700 1 -- start start 2023-06-23T01:11:40.445 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.900+0000 7f0b9a0cb700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b94070db0 0x7f0b94104f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:40.445 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.900+0000 7f0b9a0cb700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b94105460 con 0x7f0b94070db0 2023-06-23T01:11:40.445 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.900+0000 7f0b937fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b94070db0 0x7f0b94104f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:40.445 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.901+0000 7f0b937fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b94070db0 0x7f0b94104f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53172/0 (socket says 172.21.15.120:53172) 2023-06-23T01:11:40.446 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.901+0000 7f0b937fe700 1 -- 172.21.15.120:0/2460000824 learned_addr learned my addr 172.21.15.120:0/2460000824 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:40.446 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.901+0000 7f0b937fe700 1 -- 172.21.15.120:0/2460000824 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b941055a0 con 0x7f0b94070db0 2023-06-23T01:11:40.446 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.901+0000 7f0b937fe700 1 --2- 172.21.15.120:0/2460000824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b94070db0 0x7f0b94104f20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0b840098e0 tx=0x7f0b84009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9852c4726275c863 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:40.446 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.901+0000 7f0b927fc700 1 -- 172.21.15.120:0/2460000824 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b84009e80 con 0x7f0b94070db0 2023-06-23T01:11:40.446 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.901+0000 7f0b927fc700 1 -- 172.21.15.120:0/2460000824 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b8400bdc0 con 0x7f0b94070db0 2023-06-23T01:11:40.446 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.901+0000 7f0b927fc700 1 -- 172.21.15.120:0/2460000824 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b84011d80 con 0x7f0b94070db0 2023-06-23T01:11:40.446 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.902+0000 7f0b9a0cb700 1 -- 172.21.15.120:0/2460000824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b94070db0 msgr2=0x7f0b94104f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:40.446 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.902+0000 7f0b9a0cb700 1 --2- 172.21.15.120:0/2460000824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b94070db0 0x7f0b94104f20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0b840098e0 tx=0x7f0b84009250 comp rx=0 tx=0).stop 2023-06-23T01:11:40.447 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.902+0000 7f0b9a0cb700 1 -- 172.21.15.120:0/2460000824 shutdown_connections 2023-06-23T01:11:40.447 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.902+0000 7f0b9a0cb700 1 --2- 172.21.15.120:0/2460000824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b94070db0 0x7f0b94104f20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:40.447 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.903+0000 7f0b9a0cb700 1 -- 172.21.15.120:0/2460000824 >> 172.21.15.120:0/2460000824 conn(0x7f0b940fc8f0 msgr2=0x7f0b940fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:40.447 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.903+0000 7f0b9a0cb700 1 -- 172.21.15.120:0/2460000824 shutdown_connections 2023-06-23T01:11:40.447 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.903+0000 7f0b9a0cb700 1 -- 172.21.15.120:0/2460000824 wait complete. 2023-06-23T01:11:40.447 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.903+0000 7f0b9a0cb700 1 Processor -- start 2023-06-23T01:11:40.447 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.904+0000 7f0b9a0cb700 1 -- start start 2023-06-23T01:11:40.448 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.904+0000 7f0b9a0cb700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b94070db0 0x7f0b941029a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:40.448 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.904+0000 7f0b9a0cb700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b94100e20 con 0x7f0b94070db0 2023-06-23T01:11:40.448 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.904+0000 7f0b937fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b94070db0 0x7f0b941029a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:40.448 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.904+0000 7f0b937fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b94070db0 0x7f0b941029a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53176/0 (socket says 172.21.15.120:53176) 2023-06-23T01:11:40.448 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.904+0000 7f0b937fe700 1 -- 172.21.15.120:0/2080064339 learned_addr learned my addr 172.21.15.120:0/2080064339 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:40.449 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.905+0000 7f0b937fe700 1 -- 172.21.15.120:0/2080064339 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b840094f0 con 0x7f0b94070db0 2023-06-23T01:11:40.449 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.905+0000 7f0b937fe700 1 --2- 172.21.15.120:0/2080064339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b94070db0 0x7f0b941029a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0b84006d40 tx=0x7f0b84009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:40.449 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.905+0000 7f0b7bfff700 1 -- 172.21.15.120:0/2080064339 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b8400bc50 con 0x7f0b94070db0 2023-06-23T01:11:40.452 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.905+0000 7f0b7bfff700 1 -- 172.21.15.120:0/2080064339 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b84009e80 con 0x7f0b94070db0 2023-06-23T01:11:40.453 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.905+0000 7f0b9a0cb700 1 -- 172.21.15.120:0/2080064339 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b94101020 con 0x7f0b94070db0 2023-06-23T01:11:40.453 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.905+0000 7f0b7bfff700 1 -- 172.21.15.120:0/2080064339 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b8401ae10 con 0x7f0b94070db0 2023-06-23T01:11:40.453 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.905+0000 7f0b9a0cb700 1 -- 172.21.15.120:0/2080064339 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b941014b0 con 0x7f0b94070db0 2023-06-23T01:11:40.453 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.907+0000 7f0b7bfff700 1 -- 172.21.15.120:0/2080064339 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 9) v1 ==== 43232+0+0 (secure 0 0 0) 0x7f0b8401a430 con 0x7f0b94070db0 2023-06-23T01:11:40.454 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.907+0000 7f0b7bfff700 1 --2- 172.21.15.120:0/2080064339 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f0b7c036610 0x7f0b7c038ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:40.454 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.907+0000 7f0b7bfff700 1 -- 172.21.15.120:0/2080064339 --> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0b8400bc50 con 0x7f0b7c036610 2023-06-23T01:11:40.454 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.907+0000 7f0b7bfff700 1 -- 172.21.15.120:0/2080064339 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f0b8404a840 con 0x7f0b94070db0 2023-06-23T01:11:40.455 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.907+0000 7f0b92ffd700 1 -- 172.21.15.120:0/2080064339 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f0b7c036610 msgr2=0x7f0b7c038ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.120:6800/1836713308 2023-06-23T01:11:40.455 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:32.907+0000 7f0b92ffd700 1 --2- 172.21.15.120:0/2080064339 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f0b7c036610 0x7f0b7c038ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-23T01:11:40.455 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:33.108+0000 7f0b92ffd700 1 -- 172.21.15.120:0/2080064339 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f0b7c036610 msgr2=0x7f0b7c038ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.120:6800/1836713308 2023-06-23T01:11:40.455 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:33.108+0000 7f0b92ffd700 1 --2- 172.21.15.120:0/2080064339 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f0b7c036610 0x7f0b7c038ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-06-23T01:11:40.456 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:33.509+0000 7f0b92ffd700 1 -- 172.21.15.120:0/2080064339 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f0b7c036610 msgr2=0x7f0b7c038ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.120:6800/1836713308 2023-06-23T01:11:40.456 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:33.509+0000 7f0b92ffd700 1 --2- 172.21.15.120:0/2080064339 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f0b7c036610 0x7f0b7c038ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-06-23T01:11:40.456 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:34.311+0000 7f0b92ffd700 1 -- 172.21.15.120:0/2080064339 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f0b7c036610 msgr2=0x7f0b7c038ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.120:6800/1836713308 2023-06-23T01:11:40.457 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:34.311+0000 7f0b92ffd700 1 --2- 172.21.15.120:0/2080064339 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f0b7c036610 0x7f0b7c038ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-06-23T01:11:40.457 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:35.912+0000 7f0b92ffd700 1 -- 172.21.15.120:0/2080064339 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f0b7c036610 msgr2=0x7f0b7c038ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.120:6800/1836713308 2023-06-23T01:11:40.457 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:35.912+0000 7f0b92ffd700 1 --2- 172.21.15.120:0/2080064339 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f0b7c036610 0x7f0b7c038ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-06-23T01:11:40.457 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:38.969+0000 7f0b7bfff700 1 -- 172.21.15.120:0/2080064339 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mgrmap(e 10) v1 ==== 42999+0+0 (secure 0 0 0) 0x7f0b84028d70 con 0x7f0b94070db0 2023-06-23T01:11:40.458 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:38.969+0000 7f0b7bfff700 1 -- 172.21.15.120:0/2080064339 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f0b7c036610 msgr2=0x7f0b7c038ac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-23T01:11:40.458 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:38.969+0000 7f0b7bfff700 1 --2- 172.21.15.120:0/2080064339 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f0b7c036610 0x7f0b7c038ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:40.458 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:39.973+0000 7f0b7bfff700 1 -- 172.21.15.120:0/2080064339 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mgrmap(e 11) v1 ==== 43126+0+0 (secure 0 0 0) 0x7f0b84021d50 con 0x7f0b94070db0 2023-06-23T01:11:40.459 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:39.973+0000 7f0b7bfff700 1 --2- 172.21.15.120:0/2080064339 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f0b7c039f00 0x7f0b7c03c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:40.459 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:39.974+0000 7f0b7bfff700 1 -- 172.21.15.120:0/2080064339 --> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0b8400bc50 con 0x7f0b7c039f00 2023-06-23T01:11:40.459 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:39.974+0000 7f0b92ffd700 1 --2- 172.21.15.120:0/2080064339 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f0b7c039f00 0x7f0b7c03c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:40.459 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:39.975+0000 7f0b92ffd700 1 --2- 172.21.15.120:0/2080064339 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f0b7c039f00 0x7f0b7c03c2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0b880039d0 tx=0x7f0b88009350 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:40.459 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:39.976+0000 7f0b7bfff700 1 -- 172.21.15.120:0/2080064339 <== mgr.14152 v2:172.21.15.120:6800/1770912857 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f0b8400bc50 con 0x7f0b7c039f00 2023-06-23T01:11:40.460 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:39.982+0000 7f0b9a0cb700 1 -- 172.21.15.120:0/2080064339 --> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f0b9405f060 con 0x7f0b7c039f00 2023-06-23T01:11:40.460 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:39.983+0000 7f0b7bfff700 1 -- 172.21.15.120:0/2080064339 <== mgr.14152 v2:172.21.15.120:6800/1770912857 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f0b9405f060 con 0x7f0b7c039f00 2023-06-23T01:11:40.460 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:39.983+0000 7f0b9a0cb700 1 -- 172.21.15.120:0/2080064339 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f0b7c039f00 msgr2=0x7f0b7c03c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:40.460 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:39.983+0000 7f0b9a0cb700 1 --2- 172.21.15.120:0/2080064339 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f0b7c039f00 0x7f0b7c03c2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0b880039d0 tx=0x7f0b88009350 comp rx=0 tx=0).stop 2023-06-23T01:11:40.460 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:39.983+0000 7f0b9a0cb700 1 -- 172.21.15.120:0/2080064339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b94070db0 msgr2=0x7f0b941029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:40.460 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:39.983+0000 7f0b9a0cb700 1 --2- 172.21.15.120:0/2080064339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b94070db0 0x7f0b941029a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0b84006d40 tx=0x7f0b84009250 comp rx=0 tx=0).stop 2023-06-23T01:11:40.460 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:39.983+0000 7f0b9a0cb700 1 -- 172.21.15.120:0/2080064339 shutdown_connections 2023-06-23T01:11:40.461 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:39.983+0000 7f0b9a0cb700 1 --2- 172.21.15.120:0/2080064339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b94070db0 0x7f0b941029a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:40.461 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:39.983+0000 7f0b9a0cb700 1 --2- 172.21.15.120:0/2080064339 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f0b7c039f00 0x7f0b7c03c2e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:40.461 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:39.983+0000 7f0b9a0cb700 1 --2- 172.21.15.120:0/2080064339 >> [v2:172.21.15.120:6800/1836713308,v1:172.21.15.120:6801/1836713308] conn(0x7f0b7c036610 0x7f0b7c038ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:40.461 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:39.983+0000 7f0b9a0cb700 1 -- 172.21.15.120:0/2080064339 >> 172.21.15.120:0/2080064339 conn(0x7f0b940fc8f0 msgr2=0x7f0b940fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:40.461 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:39.983+0000 7f0b9a0cb700 1 -- 172.21.15.120:0/2080064339 shutdown_connections 2023-06-23T01:11:40.461 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:39.983+0000 7f0b9a0cb700 1 -- 172.21.15.120:0/2080064339 wait complete. 2023-06-23T01:11:40.461 INFO:teuthology.orchestra.run.smithi120.stdout:mgr epoch 9 is available 2023-06-23T01:11:40.461 INFO:teuthology.orchestra.run.smithi120.stdout:Generating a dashboard self-signed certificate... 2023-06-23T01:11:41.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:41 smithi120 ceph-mon[79813]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-06-23T01:11:41.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:41 smithi120 ceph-mon[79813]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-06-23T01:11:42.063 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-06-23T01:11:42.063 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.309+0000 7f8568c4d700 1 Processor -- start 2023-06-23T01:11:42.063 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.309+0000 7f8568c4d700 1 -- start start 2023-06-23T01:11:42.063 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.309+0000 7f8568c4d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8564077cb0 0x7f8564076150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:42.064 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.309+0000 7f8568c4d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85640780c0 con 0x7f8564077cb0 2023-06-23T01:11:42.064 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.309+0000 7f856259c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8564077cb0 0x7f8564076150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:42.064 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.310+0000 7f856259c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8564077cb0 0x7f8564076150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53268/0 (socket says 172.21.15.120:53268) 2023-06-23T01:11:42.064 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.310+0000 7f856259c700 1 -- 172.21.15.120:0/2949909244 learned_addr learned my addr 172.21.15.120:0/2949909244 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:42.064 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.310+0000 7f856259c700 1 -- 172.21.15.120:0/2949909244 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8564076690 con 0x7f8564077cb0 2023-06-23T01:11:42.064 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.310+0000 7f856259c700 1 --2- 172.21.15.120:0/2949909244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8564077cb0 0x7f8564076150 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f85540098e0 tx=0x7f8554009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72bd7b2c210705b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:42.065 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.310+0000 7f8561d9b700 1 -- 172.21.15.120:0/2949909244 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8554009e80 con 0x7f8564077cb0 2023-06-23T01:11:42.065 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.311+0000 7f8561d9b700 1 -- 172.21.15.120:0/2949909244 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f855400bd30 con 0x7f8564077cb0 2023-06-23T01:11:42.065 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.311+0000 7f8561d9b700 1 -- 172.21.15.120:0/2949909244 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8554011ca0 con 0x7f8564077cb0 2023-06-23T01:11:42.065 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.311+0000 7f8568c4d700 1 -- 172.21.15.120:0/2949909244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8564077cb0 msgr2=0x7f8564076150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:42.065 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.311+0000 7f8568c4d700 1 --2- 172.21.15.120:0/2949909244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8564077cb0 0x7f8564076150 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f85540098e0 tx=0x7f8554009250 comp rx=0 tx=0).stop 2023-06-23T01:11:42.070 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.311+0000 7f8568c4d700 1 -- 172.21.15.120:0/2949909244 shutdown_connections 2023-06-23T01:11:42.070 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.311+0000 7f8568c4d700 1 --2- 172.21.15.120:0/2949909244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8564077cb0 0x7f8564076150 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:42.070 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.311+0000 7f8568c4d700 1 -- 172.21.15.120:0/2949909244 >> 172.21.15.120:0/2949909244 conn(0x7f85640fd990 msgr2=0x7f85640ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:42.070 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.311+0000 7f8568c4d700 1 -- 172.21.15.120:0/2949909244 shutdown_connections 2023-06-23T01:11:42.071 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.312+0000 7f8568c4d700 1 -- 172.21.15.120:0/2949909244 wait complete. 2023-06-23T01:11:42.071 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.312+0000 7f8568c4d700 1 Processor -- start 2023-06-23T01:11:42.071 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.313+0000 7f8568c4d700 1 -- start start 2023-06-23T01:11:42.071 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.313+0000 7f8568c4d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8564077cb0 0x7f8564113110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:42.071 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.313+0000 7f8568c4d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8564113650 con 0x7f8564077cb0 2023-06-23T01:11:42.071 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.313+0000 7f856259c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8564077cb0 0x7f8564113110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:42.071 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.313+0000 7f856259c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8564077cb0 0x7f8564113110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53276/0 (socket says 172.21.15.120:53276) 2023-06-23T01:11:42.071 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.313+0000 7f856259c700 1 -- 172.21.15.120:0/1008221322 learned_addr learned my addr 172.21.15.120:0/1008221322 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:42.072 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.313+0000 7f856259c700 1 -- 172.21.15.120:0/1008221322 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85540094f0 con 0x7f8564077cb0 2023-06-23T01:11:42.072 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.314+0000 7f856259c700 1 --2- 172.21.15.120:0/1008221322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8564077cb0 0x7f8564113110 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8554011930 tx=0x7f8554009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:42.072 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.314+0000 7f855affd700 1 -- 172.21.15.120:0/1008221322 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f855401a790 con 0x7f8564077cb0 2023-06-23T01:11:42.072 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.314+0000 7f855affd700 1 -- 172.21.15.120:0/1008221322 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f855401adc0 con 0x7f8564077cb0 2023-06-23T01:11:42.072 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.314+0000 7f8568c4d700 1 -- 172.21.15.120:0/1008221322 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8564113850 con 0x7f8564077cb0 2023-06-23T01:11:42.072 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.314+0000 7f855affd700 1 -- 172.21.15.120:0/1008221322 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8554022c10 con 0x7f8564077cb0 2023-06-23T01:11:42.072 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.314+0000 7f8568c4d700 1 -- 172.21.15.120:0/1008221322 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8564113c60 con 0x7f8564077cb0 2023-06-23T01:11:42.073 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.315+0000 7f855affd700 1 -- 172.21.15.120:0/1008221322 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 12) v1 ==== 43232+0+0 (secure 0 0 0) 0x7f85540224c0 con 0x7f8564077cb0 2023-06-23T01:11:42.073 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.315+0000 7f8568c4d700 1 -- 172.21.15.120:0/1008221322 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f856404b970 con 0x7f8564077cb0 2023-06-23T01:11:42.073 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.315+0000 7f855affd700 1 --2- 172.21.15.120:0/1008221322 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f8550036610 0x7f8550038ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:42.073 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.315+0000 7f855affd700 1 -- 172.21.15.120:0/1008221322 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f8554049da0 con 0x7f8564077cb0 2023-06-23T01:11:42.073 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.316+0000 7f855bfff700 1 --2- 172.21.15.120:0/1008221322 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f8550036610 0x7f8550038ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:42.073 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.316+0000 7f855bfff700 1 --2- 172.21.15.120:0/1008221322 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f8550036610 0x7f8550038ac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f854c009990 tx=0x7f854c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:42.073 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.320+0000 7f855affd700 1 -- 172.21.15.120:0/1008221322 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f855402a8d0 con 0x7f8564077cb0 2023-06-23T01:11:42.074 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.527+0000 7f8568c4d700 1 -- 172.21.15.120:0/1008221322 --> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f85640fed40 con 0x7f8550036610 2023-06-23T01:11:42.074 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.623+0000 7f855affd700 1 -- 172.21.15.120:0/1008221322 <== mgr.14152 v2:172.21.15.120:6800/1770912857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f85640fed40 con 0x7f8550036610 2023-06-23T01:11:42.075 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.626+0000 7f8568c4d700 1 -- 172.21.15.120:0/1008221322 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f8550036610 msgr2=0x7f8550038ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:42.075 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.626+0000 7f8568c4d700 1 --2- 172.21.15.120:0/1008221322 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f8550036610 0x7f8550038ac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f854c009990 tx=0x7f854c006e40 comp rx=0 tx=0).stop 2023-06-23T01:11:42.075 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.626+0000 7f8568c4d700 1 -- 172.21.15.120:0/1008221322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8564077cb0 msgr2=0x7f8564113110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:42.075 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.626+0000 7f8568c4d700 1 --2- 172.21.15.120:0/1008221322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8564077cb0 0x7f8564113110 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8554011930 tx=0x7f8554009250 comp rx=0 tx=0).stop 2023-06-23T01:11:42.075 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.626+0000 7f8568c4d700 1 -- 172.21.15.120:0/1008221322 shutdown_connections 2023-06-23T01:11:42.076 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.626+0000 7f8568c4d700 1 --2- 172.21.15.120:0/1008221322 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f8550036610 0x7f8550038ac0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:42.076 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.626+0000 7f8568c4d700 1 --2- 172.21.15.120:0/1008221322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8564077cb0 0x7f8564113110 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:42.076 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.626+0000 7f8568c4d700 1 -- 172.21.15.120:0/1008221322 >> 172.21.15.120:0/1008221322 conn(0x7f85640fd990 msgr2=0x7f85640fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:42.076 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.626+0000 7f8568c4d700 1 -- 172.21.15.120:0/1008221322 shutdown_connections 2023-06-23T01:11:42.076 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:41.626+0000 7f8568c4d700 1 -- 172.21.15.120:0/1008221322 wait complete. 2023-06-23T01:11:42.076 INFO:teuthology.orchestra.run.smithi120.stdout:Creating initial admin user... 2023-06-23T01:11:42.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:42 smithi120 ceph-mon[79813]: mgrmap e12: x(active, since 2s) 2023-06-23T01:11:42.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:42 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:11:42.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:42 smithi120 ceph-mon[79813]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:11:42.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:42 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:11:42.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:42 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:11:43.454 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$QsRRN.Cov5YApBGmCiweDe7ngM1YteTpsk29Sh4mW3TCCgbdZk3u2", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1687482703, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-06-23T01:11:43.455 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.619+0000 7fae91c28700 1 Processor -- start 2023-06-23T01:11:43.455 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.620+0000 7fae91c28700 1 -- start start 2023-06-23T01:11:43.455 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.620+0000 7fae91c28700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae8c102210 0x7fae8c102620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:43.455 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.620+0000 7fae91c28700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8c102b60 con 0x7fae8c102210 2023-06-23T01:11:43.456 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.620+0000 7fae8b7fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae8c102210 0x7fae8c102620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:43.456 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.620+0000 7fae8b7fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae8c102210 0x7fae8c102620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60940/0 (socket says 172.21.15.120:60940) 2023-06-23T01:11:43.456 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.620+0000 7fae8b7fe700 1 -- 172.21.15.120:0/2412677163 learned_addr learned my addr 172.21.15.120:0/2412677163 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:43.457 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.621+0000 7fae8b7fe700 1 -- 172.21.15.120:0/2412677163 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae8c102ca0 con 0x7fae8c102210 2023-06-23T01:11:43.457 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.621+0000 7fae8b7fe700 1 --2- 172.21.15.120:0/2412677163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae8c102210 0x7fae8c102620 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fae7c0098e0 tx=0x7fae7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df16a37792351e03 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:43.457 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.621+0000 7fae8a7fc700 1 -- 172.21.15.120:0/2412677163 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae7c009e80 con 0x7fae8c102210 2023-06-23T01:11:43.457 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.621+0000 7fae8a7fc700 1 -- 172.21.15.120:0/2412677163 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae7c00bd30 con 0x7fae8c102210 2023-06-23T01:11:43.457 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.621+0000 7fae8a7fc700 1 -- 172.21.15.120:0/2412677163 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae7c011ca0 con 0x7fae8c102210 2023-06-23T01:11:43.458 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.622+0000 7fae91c28700 1 -- 172.21.15.120:0/2412677163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae8c102210 msgr2=0x7fae8c102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:43.458 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.622+0000 7fae91c28700 1 --2- 172.21.15.120:0/2412677163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae8c102210 0x7fae8c102620 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fae7c0098e0 tx=0x7fae7c009250 comp rx=0 tx=0).stop 2023-06-23T01:11:43.458 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.622+0000 7fae91c28700 1 -- 172.21.15.120:0/2412677163 shutdown_connections 2023-06-23T01:11:43.458 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.622+0000 7fae91c28700 1 --2- 172.21.15.120:0/2412677163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae8c102210 0x7fae8c102620 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:43.459 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.622+0000 7fae91c28700 1 -- 172.21.15.120:0/2412677163 >> 172.21.15.120:0/2412677163 conn(0x7fae8c0fd990 msgr2=0x7fae8c0ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:43.459 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.622+0000 7fae91c28700 1 -- 172.21.15.120:0/2412677163 shutdown_connections 2023-06-23T01:11:43.459 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.622+0000 7fae91c28700 1 -- 172.21.15.120:0/2412677163 wait complete. 2023-06-23T01:11:43.459 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.623+0000 7fae91c28700 1 Processor -- start 2023-06-23T01:11:43.460 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.623+0000 7fae91c28700 1 -- start start 2023-06-23T01:11:43.460 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.623+0000 7fae91c28700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae8c102210 0x7fae8c110f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:43.460 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.623+0000 7fae91c28700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8c111440 con 0x7fae8c102210 2023-06-23T01:11:43.460 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.623+0000 7fae8b7fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae8c102210 0x7fae8c110f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:43.461 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.623+0000 7fae8b7fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae8c102210 0x7fae8c110f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60946/0 (socket says 172.21.15.120:60946) 2023-06-23T01:11:43.461 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.623+0000 7fae8b7fe700 1 -- 172.21.15.120:0/169205422 learned_addr learned my addr 172.21.15.120:0/169205422 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:43.461 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.623+0000 7fae8b7fe700 1 -- 172.21.15.120:0/169205422 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7c0094f0 con 0x7fae8c102210 2023-06-23T01:11:43.461 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.624+0000 7fae8b7fe700 1 --2- 172.21.15.120:0/169205422 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae8c102210 0x7fae8c110f00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fae7c000c00 tx=0x7fae7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:43.462 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.624+0000 7fae73fff700 1 -- 172.21.15.120:0/169205422 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae7c01a760 con 0x7fae8c102210 2023-06-23T01:11:43.462 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.624+0000 7fae73fff700 1 -- 172.21.15.120:0/169205422 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae7c01ad90 con 0x7fae8c102210 2023-06-23T01:11:43.462 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.624+0000 7fae91c28700 1 -- 172.21.15.120:0/169205422 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae8c111640 con 0x7fae8c102210 2023-06-23T01:11:43.462 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.624+0000 7fae73fff700 1 -- 172.21.15.120:0/169205422 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae7c022c60 con 0x7fae8c102210 2023-06-23T01:11:43.463 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.624+0000 7fae91c28700 1 -- 172.21.15.120:0/169205422 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae8c111a50 con 0x7fae8c102210 2023-06-23T01:11:43.463 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.625+0000 7fae73fff700 1 -- 172.21.15.120:0/169205422 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 12) v1 ==== 43232+0+0 (secure 0 0 0) 0x7fae7c022570 con 0x7fae8c102210 2023-06-23T01:11:43.463 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.625+0000 7fae91c28700 1 -- 172.21.15.120:0/169205422 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae8c04b970 con 0x7fae8c102210 2023-06-23T01:11:43.464 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.626+0000 7fae73fff700 1 --2- 172.21.15.120:0/169205422 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fae740365c0 0x7fae74038a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:43.464 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.626+0000 7fae73fff700 1 -- 172.21.15.120:0/169205422 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fae7c04a700 con 0x7fae8c102210 2023-06-23T01:11:43.464 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.626+0000 7fae8affd700 1 --2- 172.21.15.120:0/169205422 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fae740365c0 0x7fae74038a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:43.465 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.626+0000 7fae8affd700 1 --2- 172.21.15.120:0/169205422 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fae740365c0 0x7fae74038a70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fae80009990 tx=0x7fae80006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:43.465 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.631+0000 7fae73fff700 1 -- 172.21.15.120:0/169205422 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fae7c02ad10 con 0x7fae8c102210 2023-06-23T01:11:43.465 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:42.859+0000 7fae91c28700 1 -- 172.21.15.120:0/169205422 --> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fae8c05ee80 con 0x7fae740365c0 2023-06-23T01:11:43.465 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.088+0000 7fae73fff700 1 -- 172.21.15.120:0/169205422 <== mgr.14152 v2:172.21.15.120:6800/1770912857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fae8c05ee80 con 0x7fae740365c0 2023-06-23T01:11:43.466 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.090+0000 7fae91c28700 1 -- 172.21.15.120:0/169205422 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fae740365c0 msgr2=0x7fae74038a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:43.466 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.090+0000 7fae91c28700 1 --2- 172.21.15.120:0/169205422 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fae740365c0 0x7fae74038a70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fae80009990 tx=0x7fae80006e40 comp rx=0 tx=0).stop 2023-06-23T01:11:43.466 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.090+0000 7fae91c28700 1 -- 172.21.15.120:0/169205422 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae8c102210 msgr2=0x7fae8c110f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:43.467 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.090+0000 7fae91c28700 1 --2- 172.21.15.120:0/169205422 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae8c102210 0x7fae8c110f00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fae7c000c00 tx=0x7fae7c009250 comp rx=0 tx=0).stop 2023-06-23T01:11:43.467 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.090+0000 7fae91c28700 1 -- 172.21.15.120:0/169205422 shutdown_connections 2023-06-23T01:11:43.467 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.090+0000 7fae91c28700 1 --2- 172.21.15.120:0/169205422 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fae740365c0 0x7fae74038a70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:43.467 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.090+0000 7fae91c28700 1 --2- 172.21.15.120:0/169205422 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae8c102210 0x7fae8c110f00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:43.468 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.090+0000 7fae91c28700 1 -- 172.21.15.120:0/169205422 >> 172.21.15.120:0/169205422 conn(0x7fae8c0fd990 msgr2=0x7fae8c0fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:43.468 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.090+0000 7fae91c28700 1 -- 172.21.15.120:0/169205422 shutdown_connections 2023-06-23T01:11:43.468 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.090+0000 7fae91c28700 1 -- 172.21.15.120:0/169205422 wait complete. 2023-06-23T01:11:43.469 INFO:teuthology.orchestra.run.smithi120.stdout:Fetching dashboard port number... 2023-06-23T01:11:44.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:43 smithi120 ceph-mon[79813]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:11:44.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:43 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:11:44.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:43 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:11:45.122 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 8443 2023-06-23T01:11:45.122 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.985+0000 7f86f352c700 1 Processor -- start 2023-06-23T01:11:45.122 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.985+0000 7f86f352c700 1 -- start start 2023-06-23T01:11:45.123 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.985+0000 7f86f352c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86ec077cd0 0x7f86ec076170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:45.123 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.985+0000 7f86f352c700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86ec0766b0 con 0x7f86ec077cd0 2023-06-23T01:11:45.123 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.985+0000 7f86f12c8700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86ec077cd0 0x7f86ec076170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:45.123 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.986+0000 7f86f12c8700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86ec077cd0 0x7f86ec076170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60950/0 (socket says 172.21.15.120:60950) 2023-06-23T01:11:45.123 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.986+0000 7f86f12c8700 1 -- 172.21.15.120:0/2695322339 learned_addr learned my addr 172.21.15.120:0/2695322339 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:45.123 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.986+0000 7f86f12c8700 1 -- 172.21.15.120:0/2695322339 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86ec0767f0 con 0x7f86ec077cd0 2023-06-23T01:11:45.124 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.986+0000 7f86f12c8700 1 --2- 172.21.15.120:0/2695322339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86ec077cd0 0x7f86ec076170 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f86e80098e0 tx=0x7f86e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff1329ac89e16aca server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:45.124 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.986+0000 7f86e3fff700 1 -- 172.21.15.120:0/2695322339 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f86e8009e80 con 0x7f86ec077cd0 2023-06-23T01:11:45.124 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.986+0000 7f86e3fff700 1 -- 172.21.15.120:0/2695322339 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86e800bd30 con 0x7f86ec077cd0 2023-06-23T01:11:45.124 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.986+0000 7f86e3fff700 1 -- 172.21.15.120:0/2695322339 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f86e8011ca0 con 0x7f86ec077cd0 2023-06-23T01:11:45.124 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.987+0000 7f86f352c700 1 -- 172.21.15.120:0/2695322339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86ec077cd0 msgr2=0x7f86ec076170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:45.124 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.987+0000 7f86f352c700 1 --2- 172.21.15.120:0/2695322339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86ec077cd0 0x7f86ec076170 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f86e80098e0 tx=0x7f86e8009250 comp rx=0 tx=0).stop 2023-06-23T01:11:45.125 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.987+0000 7f86f352c700 1 -- 172.21.15.120:0/2695322339 shutdown_connections 2023-06-23T01:11:45.125 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.987+0000 7f86f352c700 1 --2- 172.21.15.120:0/2695322339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86ec077cd0 0x7f86ec076170 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:45.125 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.987+0000 7f86f352c700 1 -- 172.21.15.120:0/2695322339 >> 172.21.15.120:0/2695322339 conn(0x7f86ec0fd990 msgr2=0x7f86ec0ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:45.125 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.987+0000 7f86f352c700 1 -- 172.21.15.120:0/2695322339 shutdown_connections 2023-06-23T01:11:45.125 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.987+0000 7f86f352c700 1 -- 172.21.15.120:0/2695322339 wait complete. 2023-06-23T01:11:45.125 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.988+0000 7f86f352c700 1 Processor -- start 2023-06-23T01:11:45.125 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.988+0000 7f86f352c700 1 -- start start 2023-06-23T01:11:45.126 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.988+0000 7f86f352c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86ec077cd0 0x7f86ec113140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:45.126 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.988+0000 7f86f352c700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86ec113680 con 0x7f86ec077cd0 2023-06-23T01:11:45.126 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.988+0000 7f86f12c8700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86ec077cd0 0x7f86ec113140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:45.126 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.988+0000 7f86f12c8700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86ec077cd0 0x7f86ec113140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60966/0 (socket says 172.21.15.120:60966) 2023-06-23T01:11:45.126 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.988+0000 7f86f12c8700 1 -- 172.21.15.120:0/1621276263 learned_addr learned my addr 172.21.15.120:0/1621276263 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:45.126 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.989+0000 7f86f12c8700 1 -- 172.21.15.120:0/1621276263 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e80094f0 con 0x7f86ec077cd0 2023-06-23T01:11:45.126 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.989+0000 7f86f12c8700 1 --2- 172.21.15.120:0/1621276263 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86ec077cd0 0x7f86ec113140 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f86e8006fd0 tx=0x7f86e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:45.127 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.989+0000 7f86e1ffb700 1 -- 172.21.15.120:0/1621276263 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f86e801a710 con 0x7f86ec077cd0 2023-06-23T01:11:45.127 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.989+0000 7f86f352c700 1 -- 172.21.15.120:0/1621276263 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86ec113880 con 0x7f86ec077cd0 2023-06-23T01:11:45.127 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.989+0000 7f86e1ffb700 1 -- 172.21.15.120:0/1621276263 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86e801ad40 con 0x7f86ec077cd0 2023-06-23T01:11:45.127 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.989+0000 7f86e1ffb700 1 -- 172.21.15.120:0/1621276263 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f86e8022c90 con 0x7f86ec077cd0 2023-06-23T01:11:45.127 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.989+0000 7f86f352c700 1 -- 172.21.15.120:0/1621276263 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86ec113c90 con 0x7f86ec077cd0 2023-06-23T01:11:45.127 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.990+0000 7f86f352c700 1 -- 172.21.15.120:0/1621276263 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86ec04b970 con 0x7f86ec077cd0 2023-06-23T01:11:45.128 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.990+0000 7f86e1ffb700 1 -- 172.21.15.120:0/1621276263 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 12) v1 ==== 43232+0+0 (secure 0 0 0) 0x7f86e8022500 con 0x7f86ec077cd0 2023-06-23T01:11:45.128 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.991+0000 7f86e1ffb700 1 --2- 172.21.15.120:0/1621276263 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f86d8036610 0x7f86d8038ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:45.128 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.991+0000 7f86e1ffb700 1 -- 172.21.15.120:0/1621276263 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f86e804a780 con 0x7f86ec077cd0 2023-06-23T01:11:45.128 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.991+0000 7f86f0ac7700 1 --2- 172.21.15.120:0/1621276263 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f86d8036610 0x7f86d8038ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:45.128 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.992+0000 7f86f0ac7700 1 --2- 172.21.15.120:0/1621276263 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f86d8036610 0x7f86d8038ac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f86dc009990 tx=0x7f86dc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:45.128 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:43.995+0000 7f86e1ffb700 1 -- 172.21.15.120:0/1621276263 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f86e80219f0 con 0x7f86ec077cd0 2023-06-23T01:11:45.128 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:44.218+0000 7f86f352c700 1 -- 172.21.15.120:0/1621276263 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f86ec1085f0 con 0x7f86ec077cd0 2023-06-23T01:11:45.129 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:44.218+0000 7f86e1ffb700 1 -- 172.21.15.120:0/1621276263 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f86e8022e60 con 0x7f86ec077cd0 2023-06-23T01:11:45.129 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:44.221+0000 7f86f352c700 1 -- 172.21.15.120:0/1621276263 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f86d8036610 msgr2=0x7f86d8038ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:45.129 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:44.221+0000 7f86f352c700 1 --2- 172.21.15.120:0/1621276263 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f86d8036610 0x7f86d8038ac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f86dc009990 tx=0x7f86dc006e40 comp rx=0 tx=0).stop 2023-06-23T01:11:45.129 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:44.221+0000 7f86f352c700 1 -- 172.21.15.120:0/1621276263 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86ec077cd0 msgr2=0x7f86ec113140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:45.129 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:44.221+0000 7f86f352c700 1 --2- 172.21.15.120:0/1621276263 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86ec077cd0 0x7f86ec113140 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f86e8006fd0 tx=0x7f86e8009250 comp rx=0 tx=0).stop 2023-06-23T01:11:45.129 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:44.221+0000 7f86f352c700 1 -- 172.21.15.120:0/1621276263 shutdown_connections 2023-06-23T01:11:45.130 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:44.221+0000 7f86f352c700 1 --2- 172.21.15.120:0/1621276263 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f86d8036610 0x7f86d8038ac0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:45.130 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:44.221+0000 7f86f352c700 1 --2- 172.21.15.120:0/1621276263 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86ec077cd0 0x7f86ec113140 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:45.130 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:44.221+0000 7f86f352c700 1 -- 172.21.15.120:0/1621276263 >> 172.21.15.120:0/1621276263 conn(0x7f86ec0fd990 msgr2=0x7f86ec0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:45.130 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:44.221+0000 7f86f352c700 1 -- 172.21.15.120:0/1621276263 shutdown_connections 2023-06-23T01:11:45.130 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:44.221+0000 7f86f352c700 1 -- 172.21.15.120:0/1621276263 wait complete. 2023-06-23T01:11:45.140 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-06-23T01:11:45.141 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout disabled 2023-06-23T01:11:45.157 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-06-23T01:11:45.158 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout inactive 2023-06-23T01:11:45.158 INFO:teuthology.orchestra.run.smithi120.stdout:firewalld.service is not enabled 2023-06-23T01:11:45.158 INFO:teuthology.orchestra.run.smithi120.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-06-23T01:11:45.160 INFO:teuthology.orchestra.run.smithi120.stdout:Ceph Dashboard is now available at: 2023-06-23T01:11:45.160 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:11:45.160 INFO:teuthology.orchestra.run.smithi120.stdout: URL: https://smithi120.front.sepia.ceph.com:8443/ 2023-06-23T01:11:45.160 INFO:teuthology.orchestra.run.smithi120.stdout: User: admin 2023-06-23T01:11:45.161 INFO:teuthology.orchestra.run.smithi120.stdout: Password: 2bh2qhnppy 2023-06-23T01:11:45.161 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:11:45.161 INFO:teuthology.orchestra.run.smithi120.stdout:Saving cluster configuration to /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/config directory 2023-06-23T01:11:45.162 INFO:teuthology.orchestra.run.smithi120.stdout:Enabling autotune for osd_memory_target 2023-06-23T01:11:45.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:44 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/1621276263' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-06-23T01:11:46.334 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.696+0000 7f4129ff8700 1 Processor -- start 2023-06-23T01:11:46.335 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.696+0000 7f4129ff8700 1 -- start start 2023-06-23T01:11:46.335 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.696+0000 7f4129ff8700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4124077cf0 0x7f4124076190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:46.335 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.696+0000 7f4129ff8700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41240766d0 con 0x7f4124077cf0 2023-06-23T01:11:46.335 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.696+0000 7f41237fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4124077cf0 0x7f4124076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:46.336 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.696+0000 7f41237fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4124077cf0 0x7f4124076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60980/0 (socket says 172.21.15.120:60980) 2023-06-23T01:11:46.336 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.696+0000 7f41237fe700 1 -- 172.21.15.120:0/2437731014 learned_addr learned my addr 172.21.15.120:0/2437731014 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:46.336 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.697+0000 7f41237fe700 1 -- 172.21.15.120:0/2437731014 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4124076810 con 0x7f4124077cf0 2023-06-23T01:11:46.336 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.697+0000 7f41237fe700 1 --2- 172.21.15.120:0/2437731014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4124077cf0 0x7f4124076190 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f41140098e0 tx=0x7f4114009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20110047780f0db6 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:46.337 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.697+0000 7f41227fc700 1 -- 172.21.15.120:0/2437731014 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4114009e80 con 0x7f4124077cf0 2023-06-23T01:11:46.337 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.697+0000 7f41227fc700 1 -- 172.21.15.120:0/2437731014 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f411400bd30 con 0x7f4124077cf0 2023-06-23T01:11:46.337 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.697+0000 7f41227fc700 1 -- 172.21.15.120:0/2437731014 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4114011ca0 con 0x7f4124077cf0 2023-06-23T01:11:46.337 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.698+0000 7f4129ff8700 1 -- 172.21.15.120:0/2437731014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4124077cf0 msgr2=0x7f4124076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:46.337 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.698+0000 7f4129ff8700 1 --2- 172.21.15.120:0/2437731014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4124077cf0 0x7f4124076190 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f41140098e0 tx=0x7f4114009250 comp rx=0 tx=0).stop 2023-06-23T01:11:46.337 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.698+0000 7f4129ff8700 1 -- 172.21.15.120:0/2437731014 shutdown_connections 2023-06-23T01:11:46.337 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.698+0000 7f4129ff8700 1 --2- 172.21.15.120:0/2437731014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4124077cf0 0x7f4124076190 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:46.338 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.698+0000 7f4129ff8700 1 -- 172.21.15.120:0/2437731014 >> 172.21.15.120:0/2437731014 conn(0x7f41240fd990 msgr2=0x7f41240ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:46.338 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.698+0000 7f4129ff8700 1 -- 172.21.15.120:0/2437731014 shutdown_connections 2023-06-23T01:11:46.338 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.698+0000 7f4129ff8700 1 -- 172.21.15.120:0/2437731014 wait complete. 2023-06-23T01:11:46.338 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.699+0000 7f4129ff8700 1 Processor -- start 2023-06-23T01:11:46.338 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.699+0000 7f4129ff8700 1 -- start start 2023-06-23T01:11:46.338 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.699+0000 7f4129ff8700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4124077cf0 0x7f4124113160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:46.338 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.699+0000 7f4129ff8700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41241136a0 con 0x7f4124077cf0 2023-06-23T01:11:46.339 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.699+0000 7f41237fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4124077cf0 0x7f4124113160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:46.339 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.699+0000 7f41237fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4124077cf0 0x7f4124113160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60992/0 (socket says 172.21.15.120:60992) 2023-06-23T01:11:46.339 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.699+0000 7f41237fe700 1 -- 172.21.15.120:0/3620963637 learned_addr learned my addr 172.21.15.120:0/3620963637 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:46.339 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.700+0000 7f41237fe700 1 -- 172.21.15.120:0/3620963637 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41140094f0 con 0x7f4124077cf0 2023-06-23T01:11:46.339 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.700+0000 7f41237fe700 1 --2- 172.21.15.120:0/3620963637 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4124077cf0 0x7f4124113160 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4114006fd0 tx=0x7f4114009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:46.339 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.700+0000 7f410bfff700 1 -- 172.21.15.120:0/3620963637 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f411401a710 con 0x7f4124077cf0 2023-06-23T01:11:46.339 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.700+0000 7f410bfff700 1 -- 172.21.15.120:0/3620963637 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f411401ad40 con 0x7f4124077cf0 2023-06-23T01:11:46.339 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.700+0000 7f4129ff8700 1 -- 172.21.15.120:0/3620963637 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41241138a0 con 0x7f4124077cf0 2023-06-23T01:11:46.340 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.700+0000 7f410bfff700 1 -- 172.21.15.120:0/3620963637 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4114022c90 con 0x7f4124077cf0 2023-06-23T01:11:46.340 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.700+0000 7f4129ff8700 1 -- 172.21.15.120:0/3620963637 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4124113cb0 con 0x7f4124077cf0 2023-06-23T01:11:46.340 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.701+0000 7f410bfff700 1 -- 172.21.15.120:0/3620963637 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 12) v1 ==== 43232+0+0 (secure 0 0 0) 0x7f4114022500 con 0x7f4124077cf0 2023-06-23T01:11:46.340 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.701+0000 7f4129ff8700 1 -- 172.21.15.120:0/3620963637 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f412404b970 con 0x7f4124077cf0 2023-06-23T01:11:46.340 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.702+0000 7f410bfff700 1 --2- 172.21.15.120:0/3620963637 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f410c036610 0x7f410c038ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:46.340 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.702+0000 7f410bfff700 1 -- 172.21.15.120:0/3620963637 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f411404a7c0 con 0x7f4124077cf0 2023-06-23T01:11:46.340 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.702+0000 7f4122ffd700 1 --2- 172.21.15.120:0/3620963637 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f410c036610 0x7f410c038ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:46.341 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.703+0000 7f4122ffd700 1 --2- 172.21.15.120:0/3620963637 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f410c036610 0x7f410c038ac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4118009990 tx=0x7f4118006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:46.341 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.707+0000 7f410bfff700 1 -- 172.21.15.120:0/3620963637 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f41140219f0 con 0x7f4124077cf0 2023-06-23T01:11:46.341 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.876+0000 7f410bfff700 1 -- 172.21.15.120:0/3620963637 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mgrmap(e 13) v1 ==== 43276+0+0 (secure 0 0 0) 0x7f411402c960 con 0x7f4124077cf0 2023-06-23T01:11:46.341 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.898+0000 7f4129ff8700 1 -- 172.21.15.120:0/3620963637 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f4124108630 con 0x7f4124077cf0 2023-06-23T01:11:46.341 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.898+0000 7f410bfff700 1 -- 172.21.15.120:0/3620963637 <== mon.0 v2:172.21.15.120:3300/0 8 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f4114021bd0 con 0x7f4124077cf0 2023-06-23T01:11:46.341 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.901+0000 7f4129ff8700 1 -- 172.21.15.120:0/3620963637 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f410c036610 msgr2=0x7f410c038ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:46.341 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.901+0000 7f4129ff8700 1 --2- 172.21.15.120:0/3620963637 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f410c036610 0x7f410c038ac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4118009990 tx=0x7f4118006e40 comp rx=0 tx=0).stop 2023-06-23T01:11:46.342 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.901+0000 7f4129ff8700 1 -- 172.21.15.120:0/3620963637 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4124077cf0 msgr2=0x7f4124113160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:46.342 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.901+0000 7f4129ff8700 1 --2- 172.21.15.120:0/3620963637 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4124077cf0 0x7f4124113160 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4114006fd0 tx=0x7f4114009250 comp rx=0 tx=0).stop 2023-06-23T01:11:46.342 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.901+0000 7f4129ff8700 1 -- 172.21.15.120:0/3620963637 shutdown_connections 2023-06-23T01:11:46.342 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.901+0000 7f4129ff8700 1 --2- 172.21.15.120:0/3620963637 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f410c036610 0x7f410c038ac0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:46.342 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.901+0000 7f4129ff8700 1 --2- 172.21.15.120:0/3620963637 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4124077cf0 0x7f4124113160 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:46.342 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.901+0000 7f4129ff8700 1 -- 172.21.15.120:0/3620963637 >> 172.21.15.120:0/3620963637 conn(0x7f41240fd990 msgr2=0x7f41240fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:46.342 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.901+0000 7f4129ff8700 1 -- 172.21.15.120:0/3620963637 shutdown_connections 2023-06-23T01:11:46.342 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:45.901+0000 7f4129ff8700 1 -- 172.21.15.120:0/3620963637 wait complete. 2023-06-23T01:11:47.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:46 smithi120 ceph-mon[79813]: mgrmap e13: x(active, since 6s) 2023-06-23T01:11:47.604 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.861+0000 7ff9e0193700 1 Processor -- start 2023-06-23T01:11:47.605 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.861+0000 7ff9e0193700 1 -- start start 2023-06-23T01:11:47.605 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.861+0000 7ff9e0193700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9d8104490 0x7ff9d81048a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:47.605 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.861+0000 7ff9e0193700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d8104de0 con 0x7ff9d8104490 2023-06-23T01:11:47.606 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.862+0000 7ff9ddf2f700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9d8104490 0x7ff9d81048a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:47.606 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.862+0000 7ff9ddf2f700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9d8104490 0x7ff9d81048a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:32770/0 (socket says 172.21.15.120:32770) 2023-06-23T01:11:47.606 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.862+0000 7ff9ddf2f700 1 -- 172.21.15.120:0/3597732777 learned_addr learned my addr 172.21.15.120:0/3597732777 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:47.606 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.862+0000 7ff9ddf2f700 1 -- 172.21.15.120:0/3597732777 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d8104f20 con 0x7ff9d8104490 2023-06-23T01:11:47.606 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.862+0000 7ff9ddf2f700 1 --2- 172.21.15.120:0/3597732777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9d8104490 0x7ff9d81048a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff9d40098e0 tx=0x7ff9d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ac36f205e7135a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:47.606 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.862+0000 7ff9dcf2d700 1 -- 172.21.15.120:0/3597732777 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9d4009e80 con 0x7ff9d8104490 2023-06-23T01:11:47.606 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.863+0000 7ff9dcf2d700 1 -- 172.21.15.120:0/3597732777 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9d400bd30 con 0x7ff9d8104490 2023-06-23T01:11:47.606 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.863+0000 7ff9dcf2d700 1 -- 172.21.15.120:0/3597732777 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9d4011ca0 con 0x7ff9d8104490 2023-06-23T01:11:47.607 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.863+0000 7ff9e0193700 1 -- 172.21.15.120:0/3597732777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9d8104490 msgr2=0x7ff9d81048a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:47.610 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.863+0000 7ff9e0193700 1 --2- 172.21.15.120:0/3597732777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9d8104490 0x7ff9d81048a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff9d40098e0 tx=0x7ff9d4009250 comp rx=0 tx=0).stop 2023-06-23T01:11:47.610 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.863+0000 7ff9e0193700 1 -- 172.21.15.120:0/3597732777 shutdown_connections 2023-06-23T01:11:47.610 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.863+0000 7ff9e0193700 1 --2- 172.21.15.120:0/3597732777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9d8104490 0x7ff9d81048a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:47.611 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.863+0000 7ff9e0193700 1 -- 172.21.15.120:0/3597732777 >> 172.21.15.120:0/3597732777 conn(0x7ff9d8077a30 msgr2=0x7ff9d81023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:47.611 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.863+0000 7ff9e0193700 1 -- 172.21.15.120:0/3597732777 shutdown_connections 2023-06-23T01:11:47.611 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.863+0000 7ff9e0193700 1 -- 172.21.15.120:0/3597732777 wait complete. 2023-06-23T01:11:47.611 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.864+0000 7ff9e0193700 1 Processor -- start 2023-06-23T01:11:47.611 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.864+0000 7ff9e0193700 1 -- start start 2023-06-23T01:11:47.611 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.864+0000 7ff9e0193700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9d8104490 0x7ff9d81130c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:47.611 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.864+0000 7ff9e0193700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d8113600 con 0x7ff9d8104490 2023-06-23T01:11:47.611 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.865+0000 7ff9ddf2f700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9d8104490 0x7ff9d81130c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:47.612 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.865+0000 7ff9ddf2f700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9d8104490 0x7ff9d81130c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:32774/0 (socket says 172.21.15.120:32774) 2023-06-23T01:11:47.612 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.865+0000 7ff9ddf2f700 1 -- 172.21.15.120:0/3186795715 learned_addr learned my addr 172.21.15.120:0/3186795715 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:47.612 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.865+0000 7ff9ddf2f700 1 -- 172.21.15.120:0/3186795715 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d40094f0 con 0x7ff9d8104490 2023-06-23T01:11:47.612 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.865+0000 7ff9ddf2f700 1 --2- 172.21.15.120:0/3186795715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9d8104490 0x7ff9d81130c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4006b00 tx=0x7ff9d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:47.612 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.865+0000 7ff9ce7fc700 1 -- 172.21.15.120:0/3186795715 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9d4018660 con 0x7ff9d8104490 2023-06-23T01:11:47.612 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.865+0000 7ff9ce7fc700 1 -- 172.21.15.120:0/3186795715 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9d4018c90 con 0x7ff9d8104490 2023-06-23T01:11:47.612 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.865+0000 7ff9ce7fc700 1 -- 172.21.15.120:0/3186795715 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9d401abf0 con 0x7ff9d8104490 2023-06-23T01:11:47.613 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.865+0000 7ff9e0193700 1 -- 172.21.15.120:0/3186795715 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9d8113800 con 0x7ff9d8104490 2023-06-23T01:11:47.613 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.866+0000 7ff9e0193700 1 -- 172.21.15.120:0/3186795715 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9d8113c10 con 0x7ff9d8104490 2023-06-23T01:11:47.613 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.867+0000 7ff9ce7fc700 1 -- 172.21.15.120:0/3186795715 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 13) v1 ==== 43276+0+0 (secure 0 0 0) 0x7ff9d401a530 con 0x7ff9d8104490 2023-06-23T01:11:47.613 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.866+0000 7ff9e0193700 1 -- 172.21.15.120:0/3186795715 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9d804b970 con 0x7ff9d8104490 2023-06-23T01:11:47.613 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.867+0000 7ff9ce7fc700 1 --2- 172.21.15.120:0/3186795715 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7ff9c4036660 0x7ff9c4038b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:47.613 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.867+0000 7ff9ce7fc700 1 -- 172.21.15.120:0/3186795715 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7ff9d404a820 con 0x7ff9d8104490 2023-06-23T01:11:47.613 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.867+0000 7ff9dd72e700 1 --2- 172.21.15.120:0/3186795715 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7ff9c4036660 0x7ff9c4038b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:47.613 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.868+0000 7ff9dd72e700 1 --2- 172.21.15.120:0/3186795715 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7ff9c4036660 0x7ff9c4038b10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8009990 tx=0x7ff9c8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:47.614 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:46.871+0000 7ff9ce7fc700 1 -- 172.21.15.120:0/3186795715 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7ff9d4020350 con 0x7ff9d8104490 2023-06-23T01:11:47.614 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:47.166+0000 7ff9e0193700 1 -- 172.21.15.120:0/3186795715 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7ff9d8114740 con 0x7ff9d8104490 2023-06-23T01:11:47.614 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:47.170+0000 7ff9ce7fc700 1 -- 172.21.15.120:0/3186795715 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) v1 ==== 153+0+0 (secure 0 0 0) 0x7ff9d402ab80 con 0x7ff9d8104490 2023-06-23T01:11:47.614 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:47.172+0000 7ff9e0193700 1 -- 172.21.15.120:0/3186795715 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7ff9c4036660 msgr2=0x7ff9c4038b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:47.614 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:47.172+0000 7ff9e0193700 1 --2- 172.21.15.120:0/3186795715 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7ff9c4036660 0x7ff9c4038b10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8009990 tx=0x7ff9c8006e40 comp rx=0 tx=0).stop 2023-06-23T01:11:47.614 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:47.172+0000 7ff9e0193700 1 -- 172.21.15.120:0/3186795715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9d8104490 msgr2=0x7ff9d81130c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:47.614 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:47.172+0000 7ff9e0193700 1 --2- 172.21.15.120:0/3186795715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9d8104490 0x7ff9d81130c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4006b00 tx=0x7ff9d4009250 comp rx=0 tx=0).stop 2023-06-23T01:11:47.615 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:47.172+0000 7ff9e0193700 1 -- 172.21.15.120:0/3186795715 shutdown_connections 2023-06-23T01:11:47.615 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:47.172+0000 7ff9e0193700 1 --2- 172.21.15.120:0/3186795715 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7ff9c4036660 0x7ff9c4038b10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:47.615 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:47.172+0000 7ff9e0193700 1 --2- 172.21.15.120:0/3186795715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9d8104490 0x7ff9d81130c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:47.615 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:47.172+0000 7ff9e0193700 1 -- 172.21.15.120:0/3186795715 >> 172.21.15.120:0/3186795715 conn(0x7ff9d8077a30 msgr2=0x7ff9d8101c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:47.616 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:47.172+0000 7ff9e0193700 1 -- 172.21.15.120:0/3186795715 shutdown_connections 2023-06-23T01:11:47.616 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2023-06-23T01:11:47.172+0000 7ff9e0193700 1 -- 172.21.15.120:0/3186795715 wait complete. 2023-06-23T01:11:47.616 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-06-23T01:11:47.616 INFO:teuthology.orchestra.run.smithi120.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-06-23T01:11:47.616 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:11:47.616 INFO:teuthology.orchestra.run.smithi120.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-06-23T01:11:47.616 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:11:47.617 INFO:teuthology.orchestra.run.smithi120.stdout:Or, if you are only running a single cluster on this host: 2023-06-23T01:11:47.617 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:11:47.617 INFO:teuthology.orchestra.run.smithi120.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-06-23T01:11:47.617 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:11:47.617 INFO:teuthology.orchestra.run.smithi120.stdout:Please consider enabling telemetry to help improve Ceph: 2023-06-23T01:11:47.617 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:11:47.617 INFO:teuthology.orchestra.run.smithi120.stdout: ceph telemetry on 2023-06-23T01:11:47.617 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:11:47.618 INFO:teuthology.orchestra.run.smithi120.stdout:For more information see: 2023-06-23T01:11:47.618 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:11:47.618 INFO:teuthology.orchestra.run.smithi120.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2023-06-23T01:11:47.618 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:11:47.618 INFO:teuthology.orchestra.run.smithi120.stdout:Bootstrap complete. 2023-06-23T01:11:47.667 INFO:tasks.cephadm:Fetching config... 2023-06-23T01:11:47.667 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2023-06-23T01:11:47.668 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-06-23T01:11:47.706 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-06-23T01:11:47.707 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2023-06-23T01:11:47.707 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-06-23T01:11:47.786 INFO:tasks.cephadm:Fetching mon keyring... 2023-06-23T01:11:47.786 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2023-06-23T01:11:47.787 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.a/keyring of=/dev/stdout 2023-06-23T01:11:47.888 INFO:tasks.cephadm:Fetching pub ssh key... 2023-06-23T01:11:47.889 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2023-06-23T01:11:47.890 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-06-23T01:11:47.964 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-06-23T01:11:47.965 DEBUG:teuthology.orchestra.run.smithi120:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD3ZA/wTqrFlLLL8uTbViox16bCHCPhvYHVeyT6FEowkewIzpFqCqQg0Zubjs/5YVA1Vz4Pw2urX6BA7T2+Ecl+luB5xuBM9qMkgzOuKZvSBaswNrgKpvD7S/JM/uNDIOx+lmAtqsS2OaBIw649paJh064pbnw2PHqpju6qNzi+ly4MFDdadGupraQ48EV7BNSsGVBGAvE0QjpAnASqvSNDTwW29qkZ3x1zz377mPh3UUCA8JJ3GClcGxKO6skDcnKHB9NvnALdq8Bzg/Hc7YfpBXFEE1stdfRnVTFahaNHc5s9JhN/6E100CXNckT8Ig4spLi+02zcYe+FwzAMuLXRJMO/B30JsFpgihbmpxs72FuYgxN9XcgTOpOhoeB/PEksa/X3ww+LpfuMH/2dOFFhDsr4UR+nY7jNWr+TI126/sNzbsK7nkPhs4xra7Xk8M/uD7B3GOU2viql0W+cOd4hNx43rkStIFPJMbghDefNGccZVfxTdvyK1kI+WyXL1VE= ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-06-23T01:11:48.102 INFO:teuthology.orchestra.run.smithi120.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD3ZA/wTqrFlLLL8uTbViox16bCHCPhvYHVeyT6FEowkewIzpFqCqQg0Zubjs/5YVA1Vz4Pw2urX6BA7T2+Ecl+luB5xuBM9qMkgzOuKZvSBaswNrgKpvD7S/JM/uNDIOx+lmAtqsS2OaBIw649paJh064pbnw2PHqpju6qNzi+ly4MFDdadGupraQ48EV7BNSsGVBGAvE0QjpAnASqvSNDTwW29qkZ3x1zz377mPh3UUCA8JJ3GClcGxKO6skDcnKHB9NvnALdq8Bzg/Hc7YfpBXFEE1stdfRnVTFahaNHc5s9JhN/6E100CXNckT8Ig4spLi+02zcYe+FwzAMuLXRJMO/B30JsFpgihbmpxs72FuYgxN9XcgTOpOhoeB/PEksa/X3ww+LpfuMH/2dOFFhDsr4UR+nY7jNWr+TI126/sNzbsK7nkPhs4xra7Xk8M/uD7B3GOU2viql0W+cOd4hNx43rkStIFPJMbghDefNGccZVfxTdvyK1kI+WyXL1VE= ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c 2023-06-23T01:11:48.131 DEBUG:teuthology.orchestra.run.smithi134:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD3ZA/wTqrFlLLL8uTbViox16bCHCPhvYHVeyT6FEowkewIzpFqCqQg0Zubjs/5YVA1Vz4Pw2urX6BA7T2+Ecl+luB5xuBM9qMkgzOuKZvSBaswNrgKpvD7S/JM/uNDIOx+lmAtqsS2OaBIw649paJh064pbnw2PHqpju6qNzi+ly4MFDdadGupraQ48EV7BNSsGVBGAvE0QjpAnASqvSNDTwW29qkZ3x1zz377mPh3UUCA8JJ3GClcGxKO6skDcnKHB9NvnALdq8Bzg/Hc7YfpBXFEE1stdfRnVTFahaNHc5s9JhN/6E100CXNckT8Ig4spLi+02zcYe+FwzAMuLXRJMO/B30JsFpgihbmpxs72FuYgxN9XcgTOpOhoeB/PEksa/X3ww+LpfuMH/2dOFFhDsr4UR+nY7jNWr+TI126/sNzbsK7nkPhs4xra7Xk8M/uD7B3GOU2viql0W+cOd4hNx43rkStIFPJMbghDefNGccZVfxTdvyK1kI+WyXL1VE= ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-06-23T01:11:48.173 INFO:teuthology.orchestra.run.smithi134.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD3ZA/wTqrFlLLL8uTbViox16bCHCPhvYHVeyT6FEowkewIzpFqCqQg0Zubjs/5YVA1Vz4Pw2urX6BA7T2+Ecl+luB5xuBM9qMkgzOuKZvSBaswNrgKpvD7S/JM/uNDIOx+lmAtqsS2OaBIw649paJh064pbnw2PHqpju6qNzi+ly4MFDdadGupraQ48EV7BNSsGVBGAvE0QjpAnASqvSNDTwW29qkZ3x1zz377mPh3UUCA8JJ3GClcGxKO6skDcnKHB9NvnALdq8Bzg/Hc7YfpBXFEE1stdfRnVTFahaNHc5s9JhN/6E100CXNckT8Ig4spLi+02zcYe+FwzAMuLXRJMO/B30JsFpgihbmpxs72FuYgxN9XcgTOpOhoeB/PEksa/X3ww+LpfuMH/2dOFFhDsr4UR+nY7jNWr+TI126/sNzbsK7nkPhs4xra7Xk8M/uD7B3GOU2viql0W+cOd4hNx43rkStIFPJMbghDefNGccZVfxTdvyK1kI+WyXL1VE= ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c 2023-06-23T01:11:48.185 DEBUG:teuthology.orchestra.run.smithi157:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD3ZA/wTqrFlLLL8uTbViox16bCHCPhvYHVeyT6FEowkewIzpFqCqQg0Zubjs/5YVA1Vz4Pw2urX6BA7T2+Ecl+luB5xuBM9qMkgzOuKZvSBaswNrgKpvD7S/JM/uNDIOx+lmAtqsS2OaBIw649paJh064pbnw2PHqpju6qNzi+ly4MFDdadGupraQ48EV7BNSsGVBGAvE0QjpAnASqvSNDTwW29qkZ3x1zz377mPh3UUCA8JJ3GClcGxKO6skDcnKHB9NvnALdq8Bzg/Hc7YfpBXFEE1stdfRnVTFahaNHc5s9JhN/6E100CXNckT8Ig4spLi+02zcYe+FwzAMuLXRJMO/B30JsFpgihbmpxs72FuYgxN9XcgTOpOhoeB/PEksa/X3ww+LpfuMH/2dOFFhDsr4UR+nY7jNWr+TI126/sNzbsK7nkPhs4xra7Xk8M/uD7B3GOU2viql0W+cOd4hNx43rkStIFPJMbghDefNGccZVfxTdvyK1kI+WyXL1VE= ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-06-23T01:11:48.226 INFO:teuthology.orchestra.run.smithi157.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD3ZA/wTqrFlLLL8uTbViox16bCHCPhvYHVeyT6FEowkewIzpFqCqQg0Zubjs/5YVA1Vz4Pw2urX6BA7T2+Ecl+luB5xuBM9qMkgzOuKZvSBaswNrgKpvD7S/JM/uNDIOx+lmAtqsS2OaBIw649paJh064pbnw2PHqpju6qNzi+ly4MFDdadGupraQ48EV7BNSsGVBGAvE0QjpAnASqvSNDTwW29qkZ3x1zz377mPh3UUCA8JJ3GClcGxKO6skDcnKHB9NvnALdq8Bzg/Hc7YfpBXFEE1stdfRnVTFahaNHc5s9JhN/6E100CXNckT8Ig4spLi+02zcYe+FwzAMuLXRJMO/B30JsFpgihbmpxs72FuYgxN9XcgTOpOhoeB/PEksa/X3ww+LpfuMH/2dOFFhDsr4UR+nY7jNWr+TI126/sNzbsK7nkPhs4xra7Xk8M/uD7B3GOU2viql0W+cOd4hNx43rkStIFPJMbghDefNGccZVfxTdvyK1kI+WyXL1VE= ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c 2023-06-23T01:11:48.238 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-06-23T01:11:48.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:48 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/3186795715' entity='client.admin' 2023-06-23T01:11:49.300 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T01:11:50.696 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.693+0000 7fa1041f3700 1 -- 172.21.15.120:0/1799472205 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0fc0fc670 msgr2=0x7fa0fc0fca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:50.696 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.693+0000 7fa1041f3700 1 --2- 172.21.15.120:0/1799472205 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0fc0fc670 0x7fa0fc0fca80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa0f80048c0 tx=0x7fa0f8009e30 comp rx=0 tx=0).stop 2023-06-23T01:11:50.697 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.693+0000 7fa1041f3700 1 -- 172.21.15.120:0/1799472205 shutdown_connections 2023-06-23T01:11:50.697 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.693+0000 7fa1041f3700 1 --2- 172.21.15.120:0/1799472205 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0fc0fc670 0x7fa0fc0fca80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:50.697 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.693+0000 7fa1041f3700 1 -- 172.21.15.120:0/1799472205 >> 172.21.15.120:0/1799472205 conn(0x7fa0fc0f7d30 msgr2=0x7fa0fc0fa160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:50.697 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.693+0000 7fa1041f3700 1 -- 172.21.15.120:0/1799472205 shutdown_connections 2023-06-23T01:11:50.697 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.693+0000 7fa1041f3700 1 -- 172.21.15.120:0/1799472205 wait complete. 2023-06-23T01:11:50.698 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.694+0000 7fa1041f3700 1 Processor -- start 2023-06-23T01:11:50.699 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.694+0000 7fa1041f3700 1 -- start start 2023-06-23T01:11:50.699 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.694+0000 7fa1041f3700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0fc0fc670 0x7fa0fc10e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:50.699 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.694+0000 7fa1041f3700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc10ef10 con 0x7fa0fc0fc670 2023-06-23T01:11:50.699 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.694+0000 7fa101f8f700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0fc0fc670 0x7fa0fc10e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:50.699 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.694+0000 7fa101f8f700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0fc0fc670 0x7fa0fc10e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:32798/0 (socket says 172.21.15.120:32798) 2023-06-23T01:11:50.699 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.694+0000 7fa101f8f700 1 -- 172.21.15.120:0/3139632417 learned_addr learned my addr 172.21.15.120:0/3139632417 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:50.700 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.695+0000 7fa101f8f700 1 -- 172.21.15.120:0/3139632417 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f8005040 con 0x7fa0fc0fc670 2023-06-23T01:11:50.700 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.695+0000 7fa101f8f700 1 --2- 172.21.15.120:0/3139632417 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0fc0fc670 0x7fa0fc10e9d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8000c00 tx=0x7fa0f8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:50.700 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.695+0000 7fa0f27fc700 1 -- 172.21.15.120:0/3139632417 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0f800e040 con 0x7fa0fc0fc670 2023-06-23T01:11:50.700 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.695+0000 7fa0f27fc700 1 -- 172.21.15.120:0/3139632417 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0f8009890 con 0x7fa0fc0fc670 2023-06-23T01:11:50.700 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.695+0000 7fa1041f3700 1 -- 172.21.15.120:0/3139632417 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0fc10f110 con 0x7fa0fc0fc670 2023-06-23T01:11:50.700 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.695+0000 7fa0f27fc700 1 -- 172.21.15.120:0/3139632417 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0f8013e90 con 0x7fa0fc0fc670 2023-06-23T01:11:50.700 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.695+0000 7fa1041f3700 1 -- 172.21.15.120:0/3139632417 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0fc10fdb0 con 0x7fa0fc0fc670 2023-06-23T01:11:50.701 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.696+0000 7fa0f27fc700 1 -- 172.21.15.120:0/3139632417 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 13) v1 ==== 43276+0+0 (secure 0 0 0) 0x7fa0f8013620 con 0x7fa0fc0fc670 2023-06-23T01:11:50.701 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.696+0000 7fa1041f3700 1 -- 172.21.15.120:0/3139632417 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0fc05f060 con 0x7fa0fc0fc670 2023-06-23T01:11:50.702 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.697+0000 7fa0f27fc700 1 --2- 172.21.15.120:0/3139632417 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fa0e80366b0 0x7fa0e8038b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:50.702 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.697+0000 7fa0f27fc700 1 -- 172.21.15.120:0/3139632417 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fa0f80495d0 con 0x7fa0fc0fc670 2023-06-23T01:11:50.702 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.697+0000 7fa10178e700 1 --2- 172.21.15.120:0/3139632417 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fa0e80366b0 0x7fa0e8038b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:50.702 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.697+0000 7fa10178e700 1 --2- 172.21.15.120:0/3139632417 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fa0e80366b0 0x7fa0e8038b60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec009990 tx=0x7fa0ec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:50.704 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.702+0000 7fa0f27fc700 1 -- 172.21.15.120:0/3139632417 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fa0f8011960 con 0x7fa0fc0fc670 2023-06-23T01:11:50.760 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:50 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:11:50.760 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:50 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:11:50.760 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:50 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi120", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:11:50.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:50 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:11:50.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:50 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:11:50.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:50 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:11:50.896 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.893+0000 7fa1041f3700 1 -- 172.21.15.120:0/3139632417 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fa0fc102340 con 0x7fa0fc0fc670 2023-06-23T01:11:50.899 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.896+0000 7fa0f27fc700 1 -- 172.21.15.120:0/3139632417 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fa0f8011370 con 0x7fa0fc0fc670 2023-06-23T01:11:50.902 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.899+0000 7fa1041f3700 1 -- 172.21.15.120:0/3139632417 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fa0e80366b0 msgr2=0x7fa0e8038b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:50.902 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.899+0000 7fa1041f3700 1 --2- 172.21.15.120:0/3139632417 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fa0e80366b0 0x7fa0e8038b60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec009990 tx=0x7fa0ec006e40 comp rx=0 tx=0).stop 2023-06-23T01:11:50.902 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.899+0000 7fa1041f3700 1 -- 172.21.15.120:0/3139632417 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0fc0fc670 msgr2=0x7fa0fc10e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:50.902 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.899+0000 7fa1041f3700 1 --2- 172.21.15.120:0/3139632417 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0fc0fc670 0x7fa0fc10e9d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8000c00 tx=0x7fa0f8009e30 comp rx=0 tx=0).stop 2023-06-23T01:11:50.902 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.899+0000 7fa1041f3700 1 -- 172.21.15.120:0/3139632417 shutdown_connections 2023-06-23T01:11:50.903 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.899+0000 7fa1041f3700 1 --2- 172.21.15.120:0/3139632417 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fa0e80366b0 0x7fa0e8038b60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:50.903 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.899+0000 7fa1041f3700 1 --2- 172.21.15.120:0/3139632417 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0fc0fc670 0x7fa0fc10e9d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:50.903 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.899+0000 7fa1041f3700 1 -- 172.21.15.120:0/3139632417 >> 172.21.15.120:0/3139632417 conn(0x7fa0fc0f7d30 msgr2=0x7fa0fc0fa160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:50.903 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.899+0000 7fa1041f3700 1 -- 172.21.15.120:0/3139632417 shutdown_connections 2023-06-23T01:11:50.903 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:50.899+0000 7fa1041f3700 1 -- 172.21.15.120:0/3139632417 wait complete. 2023-06-23T01:11:51.356 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-06-23T01:11:51.357 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-06-23T01:11:52.106 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T01:11:52.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:51 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/3139632417' entity='client.admin' 2023-06-23T01:11:52.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:51 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:11:52.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:51 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:11:53.672 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.669+0000 7feea40f7700 1 -- 172.21.15.120:0/2367360281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee9c0fc670 msgr2=0x7fee9c0fca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:53.673 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.669+0000 7feea40f7700 1 --2- 172.21.15.120:0/2367360281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee9c0fc670 0x7fee9c0fca80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fee980048c0 tx=0x7fee98009e30 comp rx=0 tx=0).stop 2023-06-23T01:11:53.673 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.669+0000 7feea40f7700 1 -- 172.21.15.120:0/2367360281 shutdown_connections 2023-06-23T01:11:53.673 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.669+0000 7feea40f7700 1 --2- 172.21.15.120:0/2367360281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee9c0fc670 0x7fee9c0fca80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:53.673 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.669+0000 7feea40f7700 1 -- 172.21.15.120:0/2367360281 >> 172.21.15.120:0/2367360281 conn(0x7fee9c0f7d30 msgr2=0x7fee9c0fa160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:53.673 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.669+0000 7feea40f7700 1 -- 172.21.15.120:0/2367360281 shutdown_connections 2023-06-23T01:11:53.674 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.669+0000 7feea40f7700 1 -- 172.21.15.120:0/2367360281 wait complete. 2023-06-23T01:11:53.675 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.670+0000 7feea40f7700 1 Processor -- start 2023-06-23T01:11:53.675 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.670+0000 7feea40f7700 1 -- start start 2023-06-23T01:11:53.675 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.670+0000 7feea40f7700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee9c0fc670 0x7fee9c10c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:53.675 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.670+0000 7feea40f7700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c10cd60 con 0x7fee9c0fc670 2023-06-23T01:11:53.675 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.670+0000 7feea1e93700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee9c0fc670 0x7fee9c10c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:53.676 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.671+0000 7feea1e93700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee9c0fc670 0x7fee9c10c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58340/0 (socket says 172.21.15.120:58340) 2023-06-23T01:11:53.676 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.671+0000 7feea1e93700 1 -- 172.21.15.120:0/1213480430 learned_addr learned my addr 172.21.15.120:0/1213480430 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:53.676 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.671+0000 7feea1e93700 1 -- 172.21.15.120:0/1213480430 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee98005040 con 0x7fee9c0fc670 2023-06-23T01:11:53.676 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.671+0000 7feea1e93700 1 --2- 172.21.15.120:0/1213480430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee9c0fc670 0x7fee9c10c820 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fee98000c00 tx=0x7fee98009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:53.676 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.671+0000 7fee927fc700 1 -- 172.21.15.120:0/1213480430 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee9800e040 con 0x7fee9c0fc670 2023-06-23T01:11:53.676 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.671+0000 7feea40f7700 1 -- 172.21.15.120:0/1213480430 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee9c10cf60 con 0x7fee9c0fc670 2023-06-23T01:11:53.677 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.671+0000 7fee927fc700 1 -- 172.21.15.120:0/1213480430 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee98009890 con 0x7fee9c0fc670 2023-06-23T01:11:53.677 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.671+0000 7fee927fc700 1 -- 172.21.15.120:0/1213480430 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee98013e90 con 0x7fee9c0fc670 2023-06-23T01:11:53.677 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.671+0000 7feea40f7700 1 -- 172.21.15.120:0/1213480430 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee9c10dc00 con 0x7fee9c0fc670 2023-06-23T01:11:53.678 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.672+0000 7fee927fc700 1 -- 172.21.15.120:0/1213480430 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 13) v1 ==== 43276+0+0 (secure 0 0 0) 0x7fee98013620 con 0x7fee9c0fc670 2023-06-23T01:11:53.678 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.672+0000 7feea40f7700 1 -- 172.21.15.120:0/1213480430 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee9c05f060 con 0x7fee9c0fc670 2023-06-23T01:11:53.678 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.673+0000 7fee927fc700 1 --2- 172.21.15.120:0/1213480430 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fee880366b0 0x7fee88038b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:53.678 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.673+0000 7fee927fc700 1 -- 172.21.15.120:0/1213480430 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fee98049570 con 0x7fee9c0fc670 2023-06-23T01:11:53.679 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.673+0000 7feea1692700 1 --2- 172.21.15.120:0/1213480430 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fee880366b0 0x7fee88038b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:53.679 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.674+0000 7feea1692700 1 --2- 172.21.15.120:0/1213480430 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fee880366b0 0x7fee88038b60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fee8c009990 tx=0x7fee8c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:53.680 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.678+0000 7fee927fc700 1 -- 172.21.15.120:0/1213480430 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fee98011960 con 0x7fee9c0fc670 2023-06-23T01:11:53.876 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.873+0000 7feea40f7700 1 -- 172.21.15.120:0/1213480430 --> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fee9c10e780 con 0x7fee880366b0 2023-06-23T01:11:53.879 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.877+0000 7fee927fc700 1 -- 172.21.15.120:0/1213480430 <== mgr.14152 v2:172.21.15.120:6800/1770912857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fee9c10e780 con 0x7fee880366b0 2023-06-23T01:11:53.882 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.879+0000 7feea40f7700 1 -- 172.21.15.120:0/1213480430 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fee880366b0 msgr2=0x7fee88038b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:53.883 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.879+0000 7feea40f7700 1 --2- 172.21.15.120:0/1213480430 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fee880366b0 0x7fee88038b60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fee8c009990 tx=0x7fee8c006e40 comp rx=0 tx=0).stop 2023-06-23T01:11:53.883 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.879+0000 7feea40f7700 1 -- 172.21.15.120:0/1213480430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee9c0fc670 msgr2=0x7fee9c10c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:53.883 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.879+0000 7feea40f7700 1 --2- 172.21.15.120:0/1213480430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee9c0fc670 0x7fee9c10c820 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fee98000c00 tx=0x7fee98009e30 comp rx=0 tx=0).stop 2023-06-23T01:11:53.884 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.879+0000 7feea40f7700 1 -- 172.21.15.120:0/1213480430 shutdown_connections 2023-06-23T01:11:53.884 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.879+0000 7feea40f7700 1 --2- 172.21.15.120:0/1213480430 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fee880366b0 0x7fee88038b60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:53.884 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.879+0000 7feea40f7700 1 --2- 172.21.15.120:0/1213480430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee9c0fc670 0x7fee9c10c820 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:53.884 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.879+0000 7feea40f7700 1 -- 172.21.15.120:0/1213480430 >> 172.21.15.120:0/1213480430 conn(0x7fee9c0f7d30 msgr2=0x7fee9c0f8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:53.884 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.879+0000 7feea40f7700 1 -- 172.21.15.120:0/1213480430 shutdown_connections 2023-06-23T01:11:53.885 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:53.879+0000 7feea40f7700 1 -- 172.21.15.120:0/1213480430 wait complete. 2023-06-23T01:11:54.365 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi134 2023-06-23T01:11:54.365 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-06-23T01:11:54.365 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/etc/ceph/ceph.conf 2023-06-23T01:11:54.382 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-06-23T01:11:54.384 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-06-23T01:11:54.439 INFO:tasks.cephadm:Adding host smithi134 to orchestrator... 2023-06-23T01:11:54.439 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph orch host add smithi134 2023-06-23T01:11:55.216 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T01:11:55.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:54 smithi120 ceph-mon[79813]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:11:55.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:54 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:11:55.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:54 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:11:55.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:54 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:11:55.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:54 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:11:55.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:54 smithi120 ceph-mon[79813]: Updating smithi120:/etc/ceph/ceph.conf 2023-06-23T01:11:56.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:55 smithi120 ceph-mon[79813]: Updating smithi120:/var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-23T01:11:56.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:55 smithi120 ceph-mon[79813]: Updating smithi120:/etc/ceph/ceph.client.admin.keyring 2023-06-23T01:11:56.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:55 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:11:56.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:55 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:11:56.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:55 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:11:56.626 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:56.623+0000 7f21e71de700 1 -- 172.21.15.120:0/110687442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21e006fdd0 msgr2=0x7f21e00701a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:56.627 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:56.623+0000 7f21e71de700 1 --2- 172.21.15.120:0/110687442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21e006fdd0 0x7f21e00701a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f21dc0048c0 tx=0x7f21dc009e30 comp rx=0 tx=0).stop 2023-06-23T01:11:56.627 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:56.623+0000 7f21e71de700 1 -- 172.21.15.120:0/110687442 shutdown_connections 2023-06-23T01:11:56.627 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:56.623+0000 7f21e71de700 1 --2- 172.21.15.120:0/110687442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21e006fdd0 0x7f21e00701a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:56.627 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:56.623+0000 7f21e71de700 1 -- 172.21.15.120:0/110687442 >> 172.21.15.120:0/110687442 conn(0x7f21e00f8200 msgr2=0x7f21e00fa610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:56.628 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:56.624+0000 7f21e71de700 1 -- 172.21.15.120:0/110687442 shutdown_connections 2023-06-23T01:11:56.628 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:56.624+0000 7f21e71de700 1 -- 172.21.15.120:0/110687442 wait complete. 2023-06-23T01:11:56.629 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:56.624+0000 7f21e71de700 1 Processor -- start 2023-06-23T01:11:56.629 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:56.624+0000 7f21e71de700 1 -- start start 2023-06-23T01:11:56.629 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:56.625+0000 7f21e71de700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21e006fdd0 0x7f21e0114b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:56.629 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:56.625+0000 7f21e71de700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e0115050 con 0x7f21e006fdd0 2023-06-23T01:11:56.630 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:56.625+0000 7f21e4f7a700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21e006fdd0 0x7f21e0114b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:56.630 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:56.625+0000 7f21e4f7a700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21e006fdd0 0x7f21e0114b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58358/0 (socket says 172.21.15.120:58358) 2023-06-23T01:11:56.630 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:56.625+0000 7f21e4f7a700 1 -- 172.21.15.120:0/1592836128 learned_addr learned my addr 172.21.15.120:0/1592836128 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:11:56.630 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:56.625+0000 7f21e4f7a700 1 -- 172.21.15.120:0/1592836128 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21dc005040 con 0x7f21e006fdd0 2023-06-23T01:11:56.630 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:56.626+0000 7f21e4f7a700 1 --2- 172.21.15.120:0/1592836128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21e006fdd0 0x7f21e0114b10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f21dc005010 tx=0x7f21dc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:56.630 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:56.626+0000 7f21d57fa700 1 -- 172.21.15.120:0/1592836128 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21dc00e040 con 0x7f21e006fdd0 2023-06-23T01:11:56.630 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:56.626+0000 7f21e71de700 1 -- 172.21.15.120:0/1592836128 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e0115250 con 0x7f21e006fdd0 2023-06-23T01:11:56.631 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:56.626+0000 7f21d57fa700 1 -- 172.21.15.120:0/1592836128 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f21dc009890 con 0x7f21e006fdd0 2023-06-23T01:11:56.631 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:56.626+0000 7f21d57fa700 1 -- 172.21.15.120:0/1592836128 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21dc013e20 con 0x7f21e006fdd0 2023-06-23T01:11:56.631 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:56.626+0000 7f21e71de700 1 -- 172.21.15.120:0/1592836128 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e01156c0 con 0x7f21e006fdd0 2023-06-23T01:11:56.632 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:56.627+0000 7f21d57fa700 1 -- 172.21.15.120:0/1592836128 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 13) v1 ==== 43276+0+0 (secure 0 0 0) 0x7f21dc0093c0 con 0x7f21e006fdd0 2023-06-23T01:11:56.632 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:56.627+0000 7f21e71de700 1 -- 172.21.15.120:0/1592836128 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e005f060 con 0x7f21e006fdd0 2023-06-23T01:11:56.632 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:56.628+0000 7f21d57fa700 1 --2- 172.21.15.120:0/1592836128 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f21c8036660 0x7f21c8038b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:11:56.633 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:56.628+0000 7f21d57fa700 1 -- 172.21.15.120:0/1592836128 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f21dc049670 con 0x7f21e006fdd0 2023-06-23T01:11:56.633 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:56.628+0000 7f21d7fff700 1 --2- 172.21.15.120:0/1592836128 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f21c8036660 0x7f21c8038b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:11:56.633 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:56.628+0000 7f21d7fff700 1 --2- 172.21.15.120:0/1592836128 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f21c8036660 0x7f21c8038b10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f21d0009990 tx=0x7f21d0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:11:56.635 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:56.632+0000 7f21d57fa700 1 -- 172.21.15.120:0/1592836128 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f21dc011db0 con 0x7f21e006fdd0 2023-06-23T01:11:56.830 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:56.827+0000 7f21e71de700 1 -- 172.21.15.120:0/1592836128 --> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi134", "target": ["mon-mgr", ""]}) v1 -- 0x7f21e004b9e0 con 0x7f21c8036660 2023-06-23T01:11:57.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:56 smithi120 ceph-mon[79813]: Updating smithi120:/var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/config/ceph.client.admin.keyring 2023-06-23T01:11:58.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:57 smithi120 ceph-mon[79813]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi134", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:11:58.614 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:58.611+0000 7f21d57fa700 1 -- 172.21.15.120:0/1592836128 <== mgr.14152 v2:172.21.15.120:6800/1770912857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f21e004b9e0 con 0x7f21c8036660 2023-06-23T01:11:58.614 INFO:teuthology.orchestra.run.smithi120.stdout:Added host 'smithi134' with addr '172.21.15.134' 2023-06-23T01:11:58.617 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:58.614+0000 7f21e71de700 1 -- 172.21.15.120:0/1592836128 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f21c8036660 msgr2=0x7f21c8038b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:58.617 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:58.614+0000 7f21e71de700 1 --2- 172.21.15.120:0/1592836128 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f21c8036660 0x7f21c8038b10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f21d0009990 tx=0x7f21d0006e40 comp rx=0 tx=0).stop 2023-06-23T01:11:58.617 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:58.614+0000 7f21e71de700 1 -- 172.21.15.120:0/1592836128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21e006fdd0 msgr2=0x7f21e0114b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:11:58.617 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:58.614+0000 7f21e71de700 1 --2- 172.21.15.120:0/1592836128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21e006fdd0 0x7f21e0114b10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f21dc005010 tx=0x7f21dc009e30 comp rx=0 tx=0).stop 2023-06-23T01:11:58.618 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:58.614+0000 7f21e71de700 1 -- 172.21.15.120:0/1592836128 shutdown_connections 2023-06-23T01:11:58.618 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:58.614+0000 7f21e71de700 1 --2- 172.21.15.120:0/1592836128 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f21c8036660 0x7f21c8038b10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:58.618 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:58.614+0000 7f21e71de700 1 --2- 172.21.15.120:0/1592836128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21e006fdd0 0x7f21e0114b10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:11:58.618 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:58.614+0000 7f21e71de700 1 -- 172.21.15.120:0/1592836128 >> 172.21.15.120:0/1592836128 conn(0x7f21e00f8200 msgr2=0x7f21e01031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:11:58.618 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:58.614+0000 7f21e71de700 1 -- 172.21.15.120:0/1592836128 shutdown_connections 2023-06-23T01:11:58.618 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:11:58.614+0000 7f21e71de700 1 -- 172.21.15.120:0/1592836128 wait complete. 2023-06-23T01:11:59.079 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph orch host ls --format=json 2023-06-23T01:11:59.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:58 smithi120 ceph-mon[79813]: Deploying cephadm binary to smithi134 2023-06-23T01:11:59.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:58 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:11:59.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:58 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:11:59.834 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T01:12:00.188 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:59 smithi120 ceph-mon[79813]: Added host smithi134 2023-06-23T01:12:00.188 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:59 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:00.188 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:11:59 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:01.234 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.230+0000 7f570af9a700 1 -- 172.21.15.120:0/2657788993 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57041026c0 msgr2=0x7f5704102a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:12:01.234 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.230+0000 7f570af9a700 1 --2- 172.21.15.120:0/2657788993 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57041026c0 0x7f5704102a90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f57000048c0 tx=0x7f5700009e30 comp rx=0 tx=0).stop 2023-06-23T01:12:01.234 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.230+0000 7f570af9a700 1 -- 172.21.15.120:0/2657788993 shutdown_connections 2023-06-23T01:12:01.234 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.230+0000 7f570af9a700 1 --2- 172.21.15.120:0/2657788993 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57041026c0 0x7f5704102a90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:01.234 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.230+0000 7f570af9a700 1 -- 172.21.15.120:0/2657788993 >> 172.21.15.120:0/2657788993 conn(0x7f57040f8200 msgr2=0x7f57040fa610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:12:01.234 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.230+0000 7f570af9a700 1 -- 172.21.15.120:0/2657788993 shutdown_connections 2023-06-23T01:12:01.235 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.230+0000 7f570af9a700 1 -- 172.21.15.120:0/2657788993 wait complete. 2023-06-23T01:12:01.235 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.231+0000 7f570af9a700 1 Processor -- start 2023-06-23T01:12:01.235 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.231+0000 7f570af9a700 1 -- start start 2023-06-23T01:12:01.236 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.231+0000 7f570af9a700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57041026c0 0x7f57041128e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:12:01.236 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.231+0000 7f570af9a700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5704112e20 con 0x7f57041026c0 2023-06-23T01:12:01.236 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.231+0000 7f5708d36700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57041026c0 0x7f57041128e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:12:01.237 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.232+0000 7f5708d36700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57041026c0 0x7f57041128e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58378/0 (socket says 172.21.15.120:58378) 2023-06-23T01:12:01.237 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.232+0000 7f5708d36700 1 -- 172.21.15.120:0/2347072199 learned_addr learned my addr 172.21.15.120:0/2347072199 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:12:01.237 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.232+0000 7f5708d36700 1 -- 172.21.15.120:0/2347072199 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5700005040 con 0x7f57041026c0 2023-06-23T01:12:01.237 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.232+0000 7f5708d36700 1 --2- 172.21.15.120:0/2347072199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57041026c0 0x7f57041128e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f570000b600 tx=0x7f5700009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:12:01.237 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.232+0000 7f56f97fa700 1 -- 172.21.15.120:0/2347072199 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f570000e040 con 0x7f57041026c0 2023-06-23T01:12:01.237 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.232+0000 7f56f97fa700 1 -- 172.21.15.120:0/2347072199 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57000098c0 con 0x7f57041026c0 2023-06-23T01:12:01.237 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.232+0000 7f56f97fa700 1 -- 172.21.15.120:0/2347072199 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5700013de0 con 0x7f57041026c0 2023-06-23T01:12:01.238 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.232+0000 7f570af9a700 1 -- 172.21.15.120:0/2347072199 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5704113020 con 0x7f57041026c0 2023-06-23T01:12:01.238 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.232+0000 7f570af9a700 1 -- 172.21.15.120:0/2347072199 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5704113490 con 0x7f57041026c0 2023-06-23T01:12:01.238 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.233+0000 7f570af9a700 1 -- 172.21.15.120:0/2347072199 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f570405f060 con 0x7f57041026c0 2023-06-23T01:12:01.238 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.233+0000 7f56f97fa700 1 -- 172.21.15.120:0/2347072199 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 13) v1 ==== 43276+0+0 (secure 0 0 0) 0x7f57000135b0 con 0x7f57041026c0 2023-06-23T01:12:01.239 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.234+0000 7f56f97fa700 1 --2- 172.21.15.120:0/2347072199 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f56ec036660 0x7f56ec038b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:12:01.239 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.234+0000 7f56f97fa700 1 -- 172.21.15.120:0/2347072199 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f5700049600 con 0x7f57041026c0 2023-06-23T01:12:01.239 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.234+0000 7f56fbfff700 1 --2- 172.21.15.120:0/2347072199 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f56ec036660 0x7f56ec038b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:12:01.239 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.235+0000 7f56fbfff700 1 --2- 172.21.15.120:0/2347072199 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f56ec036660 0x7f56ec038b10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f56f4009990 tx=0x7f56f4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:12:01.241 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.239+0000 7f56f97fa700 1 -- 172.21.15.120:0/2347072199 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f5700023300 con 0x7f57041026c0 2023-06-23T01:12:01.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:00 smithi120 ceph-mon[79813]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:01.439 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.436+0000 7f570af9a700 1 -- 172.21.15.120:0/2347072199 --> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f570404b9e0 con 0x7f56ec036660 2023-06-23T01:12:01.440 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.438+0000 7f56f97fa700 1 -- 172.21.15.120:0/2347072199 <== mgr.14152 v2:172.21.15.120:6800/1770912857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7f570404b9e0 con 0x7f56ec036660 2023-06-23T01:12:01.441 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:12:01.441 INFO:teuthology.orchestra.run.smithi120.stdout:[{"addr": "172.21.15.120", "hostname": "smithi120", "labels": [], "status": ""}, {"addr": "172.21.15.134", "hostname": "smithi134", "labels": [], "status": ""}] 2023-06-23T01:12:01.443 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.440+0000 7f570af9a700 1 -- 172.21.15.120:0/2347072199 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f56ec036660 msgr2=0x7f56ec038b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:12:01.443 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.440+0000 7f570af9a700 1 --2- 172.21.15.120:0/2347072199 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f56ec036660 0x7f56ec038b10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f56f4009990 tx=0x7f56f4006e40 comp rx=0 tx=0).stop 2023-06-23T01:12:01.443 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.440+0000 7f570af9a700 1 -- 172.21.15.120:0/2347072199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57041026c0 msgr2=0x7f57041128e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:12:01.443 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.440+0000 7f570af9a700 1 --2- 172.21.15.120:0/2347072199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57041026c0 0x7f57041128e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f570000b600 tx=0x7f5700009e30 comp rx=0 tx=0).stop 2023-06-23T01:12:01.443 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.440+0000 7f570af9a700 1 -- 172.21.15.120:0/2347072199 shutdown_connections 2023-06-23T01:12:01.444 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.440+0000 7f570af9a700 1 --2- 172.21.15.120:0/2347072199 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f56ec036660 0x7f56ec038b10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:01.444 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.440+0000 7f570af9a700 1 --2- 172.21.15.120:0/2347072199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57041026c0 0x7f57041128e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:01.444 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.440+0000 7f570af9a700 1 -- 172.21.15.120:0/2347072199 >> 172.21.15.120:0/2347072199 conn(0x7f57040f8200 msgr2=0x7f57040f8e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:12:01.444 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.440+0000 7f570af9a700 1 -- 172.21.15.120:0/2347072199 shutdown_connections 2023-06-23T01:12:01.444 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:01.440+0000 7f570af9a700 1 -- 172.21.15.120:0/2347072199 wait complete. 2023-06-23T01:12:01.889 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi157 2023-06-23T01:12:01.889 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-06-23T01:12:01.890 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/etc/ceph/ceph.conf 2023-06-23T01:12:01.907 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-06-23T01:12:01.908 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-06-23T01:12:01.964 INFO:tasks.cephadm:Adding host smithi157 to orchestrator... 2023-06-23T01:12:01.965 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph orch host add smithi157 2023-06-23T01:12:02.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:02 smithi120 ceph-mon[79813]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:02.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:02 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:02.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:02 smithi120 ceph-mon[79813]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-23T01:12:02.720 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T01:12:04.124 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:04.119+0000 7f020078e700 1 -- 172.21.15.120:0/812507985 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01f81026c0 msgr2=0x7f01f8102a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:12:04.125 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:04.119+0000 7f020078e700 1 --2- 172.21.15.120:0/812507985 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01f81026c0 0x7f01f8102a90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f01f40048c0 tx=0x7f01f4009e30 comp rx=0 tx=0).stop 2023-06-23T01:12:04.125 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:04.119+0000 7f020078e700 1 -- 172.21.15.120:0/812507985 shutdown_connections 2023-06-23T01:12:04.125 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:04.119+0000 7f020078e700 1 --2- 172.21.15.120:0/812507985 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01f81026c0 0x7f01f8102a90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:04.126 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:04.119+0000 7f020078e700 1 -- 172.21.15.120:0/812507985 >> 172.21.15.120:0/812507985 conn(0x7f01f80f8200 msgr2=0x7f01f80fa610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:12:04.126 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:04.120+0000 7f020078e700 1 -- 172.21.15.120:0/812507985 shutdown_connections 2023-06-23T01:12:04.126 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:04.120+0000 7f020078e700 1 -- 172.21.15.120:0/812507985 wait complete. 2023-06-23T01:12:04.126 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:04.120+0000 7f020078e700 1 Processor -- start 2023-06-23T01:12:04.126 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:04.120+0000 7f020078e700 1 -- start start 2023-06-23T01:12:04.126 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:04.121+0000 7f020078e700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01f81026c0 0x7f01f8114b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:12:04.126 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:04.121+0000 7f020078e700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f8115070 con 0x7f01f81026c0 2023-06-23T01:12:04.126 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:04.121+0000 7f01fe52a700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01f81026c0 0x7f01f8114b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:12:04.127 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:04.121+0000 7f01fe52a700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01f81026c0 0x7f01f8114b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45856/0 (socket says 172.21.15.120:45856) 2023-06-23T01:12:04.127 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:04.121+0000 7f01fe52a700 1 -- 172.21.15.120:0/3755676375 learned_addr learned my addr 172.21.15.120:0/3755676375 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:12:04.127 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:04.121+0000 7f01fe52a700 1 -- 172.21.15.120:0/3755676375 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f4005040 con 0x7f01f81026c0 2023-06-23T01:12:04.127 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:04.122+0000 7f01fe52a700 1 --2- 172.21.15.120:0/3755676375 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01f81026c0 0x7f01f8114b30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f01f400b600 tx=0x7f01f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:12:04.128 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:04.122+0000 7f01eeffd700 1 -- 172.21.15.120:0/3755676375 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01f400e040 con 0x7f01f81026c0 2023-06-23T01:12:04.129 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:04.122+0000 7f01eeffd700 1 -- 172.21.15.120:0/3755676375 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01f40098c0 con 0x7f01f81026c0 2023-06-23T01:12:04.129 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:04.122+0000 7f01eeffd700 1 -- 172.21.15.120:0/3755676375 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01f4013de0 con 0x7f01f81026c0 2023-06-23T01:12:04.129 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:04.122+0000 7f020078e700 1 -- 172.21.15.120:0/3755676375 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f8115270 con 0x7f01f81026c0 2023-06-23T01:12:04.130 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:04.122+0000 7f020078e700 1 -- 172.21.15.120:0/3755676375 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f81156e0 con 0x7f01f81026c0 2023-06-23T01:12:04.130 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:04.123+0000 7f01eeffd700 1 -- 172.21.15.120:0/3755676375 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 13) v1 ==== 43276+0+0 (secure 0 0 0) 0x7f01f40135b0 con 0x7f01f81026c0 2023-06-23T01:12:04.130 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:04.123+0000 7f020078e700 1 -- 172.21.15.120:0/3755676375 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01f805f060 con 0x7f01f81026c0 2023-06-23T01:12:04.130 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:04.123+0000 7f01eeffd700 1 --2- 172.21.15.120:0/3755676375 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f01e40366b0 0x7f01e4038b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:12:04.131 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:04.123+0000 7f01eeffd700 1 -- 172.21.15.120:0/3755676375 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f01f4049540 con 0x7f01f81026c0 2023-06-23T01:12:04.131 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:04.124+0000 7f01fdd29700 1 --2- 172.21.15.120:0/3755676375 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f01e40366b0 0x7f01e4038b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:12:04.131 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:04.124+0000 7f01fdd29700 1 --2- 172.21.15.120:0/3755676375 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f01e40366b0 0x7f01e4038b60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f01e8009990 tx=0x7f01e8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:12:04.132 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:04.128+0000 7f01eeffd700 1 -- 172.21.15.120:0/3755676375 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f01f4023300 con 0x7f01f81026c0 2023-06-23T01:12:04.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:04 smithi120 ceph-mon[79813]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:04.332 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:04.329+0000 7f020078e700 1 -- 172.21.15.120:0/3755676375 --> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi157", "target": ["mon-mgr", ""]}) v1 -- 0x7f01f804b970 con 0x7f01e40366b0 2023-06-23T01:12:05.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:05 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:05.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:05 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:05.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:05 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:05.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:05 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:05.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:05 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi134", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:12:05.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:05 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:05.512 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:05 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:12:05.512 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:05 smithi120 ceph-mon[79813]: Updating smithi134:/etc/ceph/ceph.conf 2023-06-23T01:12:05.512 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:05 smithi120 ceph-mon[79813]: Updating smithi134:/var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-23T01:12:05.513 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:05 smithi120 ceph-mon[79813]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi157", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:12:05.513 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:05 smithi120 ceph-mon[79813]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2023-06-23T01:12:05.513 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:05 smithi120 ceph-mon[79813]: Updating smithi134:/var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/config/ceph.client.admin.keyring 2023-06-23T01:12:05.513 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:05 smithi120 ceph-mon[79813]: Deploying cephadm binary to smithi157 2023-06-23T01:12:05.514 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:05 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:05.514 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:05 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:05.514 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:05 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:06.207 INFO:teuthology.orchestra.run.smithi120.stdout:Added host 'smithi157' with addr '172.21.15.157' 2023-06-23T01:12:06.208 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:06.203+0000 7f01eeffd700 1 -- 172.21.15.120:0/3755676375 <== mgr.14152 v2:172.21.15.120:6800/1770912857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f01f804b970 con 0x7f01e40366b0 2023-06-23T01:12:06.210 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:06.206+0000 7f020078e700 1 -- 172.21.15.120:0/3755676375 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f01e40366b0 msgr2=0x7f01e4038b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:12:06.210 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:06.206+0000 7f020078e700 1 --2- 172.21.15.120:0/3755676375 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f01e40366b0 0x7f01e4038b60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f01e8009990 tx=0x7f01e8006e40 comp rx=0 tx=0).stop 2023-06-23T01:12:06.210 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:06.206+0000 7f020078e700 1 -- 172.21.15.120:0/3755676375 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01f81026c0 msgr2=0x7f01f8114b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:12:06.211 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:06.206+0000 7f020078e700 1 --2- 172.21.15.120:0/3755676375 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01f81026c0 0x7f01f8114b30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f01f400b600 tx=0x7f01f4009e30 comp rx=0 tx=0).stop 2023-06-23T01:12:06.211 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:06.206+0000 7f020078e700 1 -- 172.21.15.120:0/3755676375 shutdown_connections 2023-06-23T01:12:06.211 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:06.206+0000 7f020078e700 1 --2- 172.21.15.120:0/3755676375 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f01e40366b0 0x7f01e4038b60 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:06.211 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:06.206+0000 7f020078e700 1 --2- 172.21.15.120:0/3755676375 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01f81026c0 0x7f01f8114b30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:06.211 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:06.206+0000 7f020078e700 1 -- 172.21.15.120:0/3755676375 >> 172.21.15.120:0/3755676375 conn(0x7f01f80f8200 msgr2=0x7f01f8104810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:12:06.211 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:06.206+0000 7f020078e700 1 -- 172.21.15.120:0/3755676375 shutdown_connections 2023-06-23T01:12:06.211 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:06.206+0000 7f020078e700 1 -- 172.21.15.120:0/3755676375 wait complete. 2023-06-23T01:12:06.391 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:06 smithi120 ceph-mon[79813]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:06.679 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph orch host ls --format=json 2023-06-23T01:12:07.432 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T01:12:07.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:07 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:07.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:07 smithi120 ceph-mon[79813]: Added host smithi157 2023-06-23T01:12:07.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:07 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:12:07.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:07 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:08.600 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:08 smithi120 ceph-mon[79813]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:08.600 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:08 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:08.824 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:08.819+0000 7f118e6ab700 1 -- 172.21.15.120:0/942063343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11881048c0 msgr2=0x7f1188104c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:12:08.824 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:08.819+0000 7f118e6ab700 1 --2- 172.21.15.120:0/942063343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11881048c0 0x7f1188104c90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f11780048c0 tx=0x7f1178009e30 comp rx=0 tx=0).stop 2023-06-23T01:12:08.825 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:08.820+0000 7f118e6ab700 1 -- 172.21.15.120:0/942063343 shutdown_connections 2023-06-23T01:12:08.825 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:08.820+0000 7f118e6ab700 1 --2- 172.21.15.120:0/942063343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11881048c0 0x7f1188104c90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:08.825 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:08.820+0000 7f118e6ab700 1 -- 172.21.15.120:0/942063343 >> 172.21.15.120:0/942063343 conn(0x7f11880fa420 msgr2=0x7f11880fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:12:08.826 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:08.820+0000 7f118e6ab700 1 -- 172.21.15.120:0/942063343 shutdown_connections 2023-06-23T01:12:08.826 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:08.820+0000 7f118e6ab700 1 -- 172.21.15.120:0/942063343 wait complete. 2023-06-23T01:12:08.829 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:08.820+0000 7f118e6ab700 1 Processor -- start 2023-06-23T01:12:08.830 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:08.821+0000 7f118e6ab700 1 -- start start 2023-06-23T01:12:08.830 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:08.821+0000 7f118e6ab700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11881048c0 0x7f1188114ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:12:08.831 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:08.821+0000 7f118e6ab700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1188115010 con 0x7f11881048c0 2023-06-23T01:12:08.831 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:08.821+0000 7f1187fff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11881048c0 0x7f1188114ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:12:08.831 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:08.821+0000 7f1187fff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11881048c0 0x7f1188114ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45874/0 (socket says 172.21.15.120:45874) 2023-06-23T01:12:08.831 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:08.821+0000 7f1187fff700 1 -- 172.21.15.120:0/2753102322 learned_addr learned my addr 172.21.15.120:0/2753102322 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:12:08.832 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:08.822+0000 7f1187fff700 1 -- 172.21.15.120:0/2753102322 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1178005040 con 0x7f11881048c0 2023-06-23T01:12:08.832 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:08.822+0000 7f1187fff700 1 --2- 172.21.15.120:0/2753102322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11881048c0 0x7f1188114ad0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f117800b600 tx=0x7f1178009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:12:08.832 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:08.822+0000 7f1184ff9700 1 -- 172.21.15.120:0/2753102322 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f117800e040 con 0x7f11881048c0 2023-06-23T01:12:08.832 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:08.822+0000 7f1184ff9700 1 -- 172.21.15.120:0/2753102322 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11780098c0 con 0x7f11881048c0 2023-06-23T01:12:08.833 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:08.822+0000 7f1184ff9700 1 -- 172.21.15.120:0/2753102322 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1178013de0 con 0x7f11881048c0 2023-06-23T01:12:08.833 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:08.822+0000 7f118e6ab700 1 -- 172.21.15.120:0/2753102322 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1188115210 con 0x7f11881048c0 2023-06-23T01:12:08.833 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:08.822+0000 7f118e6ab700 1 -- 172.21.15.120:0/2753102322 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f118810e990 con 0x7f11881048c0 2023-06-23T01:12:08.834 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:08.823+0000 7f1184ff9700 1 -- 172.21.15.120:0/2753102322 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 13) v1 ==== 43276+0+0 (secure 0 0 0) 0x7f11780135b0 con 0x7f11881048c0 2023-06-23T01:12:08.834 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:08.823+0000 7f118e6ab700 1 -- 172.21.15.120:0/2753102322 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f118805f060 con 0x7f11881048c0 2023-06-23T01:12:08.834 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:08.824+0000 7f1184ff9700 1 --2- 172.21.15.120:0/2753102322 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f11700366b0 0x7f1170038b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:12:08.834 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:08.824+0000 7f1184ff9700 1 -- 172.21.15.120:0/2753102322 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f1178049540 con 0x7f11881048c0 2023-06-23T01:12:08.835 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:08.824+0000 7f11877fe700 1 --2- 172.21.15.120:0/2753102322 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f11700366b0 0x7f1170038b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:12:08.835 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:08.824+0000 7f11877fe700 1 --2- 172.21.15.120:0/2753102322 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f11700366b0 0x7f1170038b60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f117c009990 tx=0x7f117c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:12:08.835 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:08.829+0000 7f1184ff9700 1 -- 172.21.15.120:0/2753102322 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f1178023300 con 0x7f11881048c0 2023-06-23T01:12:09.030 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:09.026+0000 7f118e6ab700 1 -- 172.21.15.120:0/2753102322 --> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f11881147a0 con 0x7f11700366b0 2023-06-23T01:12:09.031 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:09.027+0000 7f1184ff9700 1 -- 172.21.15.120:0/2753102322 <== mgr.14152 v2:172.21.15.120:6800/1770912857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+241 (secure 0 0 0) 0x7f11881147a0 con 0x7f11700366b0 2023-06-23T01:12:09.031 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:12:09.031 INFO:teuthology.orchestra.run.smithi120.stdout:[{"addr": "172.21.15.120", "hostname": "smithi120", "labels": [], "status": ""}, {"addr": "172.21.15.134", "hostname": "smithi134", "labels": [], "status": ""}, {"addr": "172.21.15.157", "hostname": "smithi157", "labels": [], "status": ""}] 2023-06-23T01:12:09.034 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:09.029+0000 7f118e6ab700 1 -- 172.21.15.120:0/2753102322 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f11700366b0 msgr2=0x7f1170038b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:12:09.034 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:09.029+0000 7f118e6ab700 1 --2- 172.21.15.120:0/2753102322 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f11700366b0 0x7f1170038b60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f117c009990 tx=0x7f117c006e40 comp rx=0 tx=0).stop 2023-06-23T01:12:09.035 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:09.030+0000 7f118e6ab700 1 -- 172.21.15.120:0/2753102322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11881048c0 msgr2=0x7f1188114ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:12:09.035 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:09.030+0000 7f118e6ab700 1 --2- 172.21.15.120:0/2753102322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11881048c0 0x7f1188114ad0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f117800b600 tx=0x7f1178009e30 comp rx=0 tx=0).stop 2023-06-23T01:12:09.035 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:09.030+0000 7f118e6ab700 1 -- 172.21.15.120:0/2753102322 shutdown_connections 2023-06-23T01:12:09.035 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:09.030+0000 7f118e6ab700 1 --2- 172.21.15.120:0/2753102322 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f11700366b0 0x7f1170038b60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:09.036 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:09.030+0000 7f118e6ab700 1 --2- 172.21.15.120:0/2753102322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11881048c0 0x7f1188114ad0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:09.036 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:09.030+0000 7f118e6ab700 1 -- 172.21.15.120:0/2753102322 >> 172.21.15.120:0/2753102322 conn(0x7f11880fa420 msgr2=0x7f11880fb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:12:09.036 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:09.030+0000 7f118e6ab700 1 -- 172.21.15.120:0/2753102322 shutdown_connections 2023-06-23T01:12:09.037 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:09.030+0000 7f118e6ab700 1 -- 172.21.15.120:0/2753102322 wait complete. 2023-06-23T01:12:09.488 INFO:tasks.cephadm:Setting crush tunables to default 2023-06-23T01:12:09.488 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph osd crush tunables default 2023-06-23T01:12:09.760 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:09 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:10.240 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T01:12:10.760 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:10 smithi120 ceph-mon[79813]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:10.760 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:10 smithi120 ceph-mon[79813]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-23T01:12:11.744 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:11.740+0000 7fdcf1f11700 1 -- 172.21.15.120:0/2515537907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdcec0fd680 msgr2=0x7fdcec0fda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:12:11.744 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:11.740+0000 7fdcf1f11700 1 --2- 172.21.15.120:0/2515537907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdcec0fd680 0x7fdcec0fda50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0048c0 tx=0x7fdcdc009e30 comp rx=0 tx=0).stop 2023-06-23T01:12:11.744 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:11.740+0000 7fdcf1f11700 1 -- 172.21.15.120:0/2515537907 shutdown_connections 2023-06-23T01:12:11.745 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:11.740+0000 7fdcf1f11700 1 --2- 172.21.15.120:0/2515537907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdcec0fd680 0x7fdcec0fda50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:11.745 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:11.740+0000 7fdcf1f11700 1 -- 172.21.15.120:0/2515537907 >> 172.21.15.120:0/2515537907 conn(0x7fdcec0713d0 msgr2=0x7fdcec0717d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:12:11.745 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:11.740+0000 7fdcf1f11700 1 -- 172.21.15.120:0/2515537907 shutdown_connections 2023-06-23T01:12:11.745 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:11.740+0000 7fdcf1f11700 1 -- 172.21.15.120:0/2515537907 wait complete. 2023-06-23T01:12:11.746 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:11.741+0000 7fdcf1f11700 1 Processor -- start 2023-06-23T01:12:11.746 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:11.741+0000 7fdcf1f11700 1 -- start start 2023-06-23T01:12:11.746 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:11.741+0000 7fdcf1f11700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdcec0fd680 0x7fdcec114a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:12:11.746 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:11.741+0000 7fdcf1f11700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec114fd0 con 0x7fdcec0fd680 2023-06-23T01:12:11.747 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:11.741+0000 7fdceb7fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdcec0fd680 0x7fdcec114a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:12:11.747 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:11.742+0000 7fdceb7fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdcec0fd680 0x7fdcec114a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55782/0 (socket says 172.21.15.120:55782) 2023-06-23T01:12:11.747 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:11.742+0000 7fdceb7fe700 1 -- 172.21.15.120:0/2579866356 learned_addr learned my addr 172.21.15.120:0/2579866356 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:12:11.747 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:11.742+0000 7fdceb7fe700 1 -- 172.21.15.120:0/2579866356 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcdc005040 con 0x7fdcec0fd680 2023-06-23T01:12:11.747 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:11.742+0000 7fdceb7fe700 1 --2- 172.21.15.120:0/2579866356 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdcec0fd680 0x7fdcec114a90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc00b600 tx=0x7fdcdc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:12:11.748 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:11.742+0000 7fdcd3fff700 1 -- 172.21.15.120:0/2579866356 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdcdc00e040 con 0x7fdcec0fd680 2023-06-23T01:12:11.748 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:11.742+0000 7fdcd3fff700 1 -- 172.21.15.120:0/2579866356 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdcdc0098c0 con 0x7fdcec0fd680 2023-06-23T01:12:11.748 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:11.742+0000 7fdcf1f11700 1 -- 172.21.15.120:0/2579866356 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcec1151d0 con 0x7fdcec0fd680 2023-06-23T01:12:11.748 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:11.743+0000 7fdcd3fff700 1 -- 172.21.15.120:0/2579866356 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdcdc013de0 con 0x7fdcec0fd680 2023-06-23T01:12:11.748 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:11.743+0000 7fdcf1f11700 1 -- 172.21.15.120:0/2579866356 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcec10e950 con 0x7fdcec0fd680 2023-06-23T01:12:11.749 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:11.744+0000 7fdcd3fff700 1 -- 172.21.15.120:0/2579866356 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 13) v1 ==== 43276+0+0 (secure 0 0 0) 0x7fdcdc0135b0 con 0x7fdcec0fd680 2023-06-23T01:12:11.749 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:11.744+0000 7fdcf1f11700 1 -- 172.21.15.120:0/2579866356 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcec05f060 con 0x7fdcec0fd680 2023-06-23T01:12:11.749 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:11.744+0000 7fdcd3fff700 1 --2- 172.21.15.120:0/2579866356 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fdcd4036660 0x7fdcd4038b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:12:11.750 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:11.744+0000 7fdcd3fff700 1 -- 172.21.15.120:0/2579866356 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fdcdc049600 con 0x7fdcec0fd680 2023-06-23T01:12:11.750 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:11.745+0000 7fdceaffd700 1 --2- 172.21.15.120:0/2579866356 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fdcd4036660 0x7fdcd4038b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:12:11.750 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:11.745+0000 7fdceaffd700 1 --2- 172.21.15.120:0/2579866356 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fdcd4036660 0x7fdcd4038b10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdce0009990 tx=0x7fdce0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:12:11.753 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:11.749+0000 7fdcd3fff700 1 -- 172.21.15.120:0/2579866356 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fdcdc023300 con 0x7fdcec0fd680 2023-06-23T01:12:11.953 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:11.949+0000 7fdcf1f11700 1 -- 172.21.15.120:0/2579866356 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fdcec110070 con 0x7fdcec0fd680 2023-06-23T01:12:12.896 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:12.892+0000 7fdcd3fff700 1 -- 172.21.15.120:0/2579866356 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fdcdc011b60 con 0x7fdcec0fd680 2023-06-23T01:12:12.899 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:12.895+0000 7fdcf1f11700 1 -- 172.21.15.120:0/2579866356 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fdcd4036660 msgr2=0x7fdcd4038b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:12:12.899 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:12.895+0000 7fdcf1f11700 1 --2- 172.21.15.120:0/2579866356 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fdcd4036660 0x7fdcd4038b10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdce0009990 tx=0x7fdce0006e40 comp rx=0 tx=0).stop 2023-06-23T01:12:12.899 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:12.895+0000 7fdcf1f11700 1 -- 172.21.15.120:0/2579866356 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdcec0fd680 msgr2=0x7fdcec114a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:12:12.899 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:12.895+0000 7fdcf1f11700 1 --2- 172.21.15.120:0/2579866356 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdcec0fd680 0x7fdcec114a90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc00b600 tx=0x7fdcdc009e30 comp rx=0 tx=0).stop 2023-06-23T01:12:12.900 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:12.895+0000 7fdcf1f11700 1 -- 172.21.15.120:0/2579866356 shutdown_connections 2023-06-23T01:12:12.900 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:12.895+0000 7fdcf1f11700 1 --2- 172.21.15.120:0/2579866356 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fdcd4036660 0x7fdcd4038b10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:12.900 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:12.895+0000 7fdcf1f11700 1 --2- 172.21.15.120:0/2579866356 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdcec0fd680 0x7fdcec114a90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:12.900 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:12.895+0000 7fdcf1f11700 1 -- 172.21.15.120:0/2579866356 >> 172.21.15.120:0/2579866356 conn(0x7fdcec0713d0 msgr2=0x7fdcec103330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:12:12.900 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:12.895+0000 7fdcf1f11700 1 -- 172.21.15.120:0/2579866356 shutdown_connections 2023-06-23T01:12:12.900 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:12.895+0000 7fdcf1f11700 1 -- 172.21.15.120:0/2579866356 wait complete. 2023-06-23T01:12:12.901 INFO:teuthology.orchestra.run.smithi120.stderr:adjusted tunables profile to default 2023-06-23T01:12:13.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:12 smithi120 ceph-mon[79813]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:13.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:12 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:13.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:12 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:13.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:12 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:13.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:12 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:13.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:12 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi157", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:12:13.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:12 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:13.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:12 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:12:13.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:12 smithi120 ceph-mon[79813]: Updating smithi157:/etc/ceph/ceph.conf 2023-06-23T01:12:13.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:12 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/2579866356' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-06-23T01:12:13.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:12 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:13.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:12 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:13.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:12 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:13.301 INFO:tasks.cephadm:Adding mon.a on smithi120 2023-06-23T01:12:13.301 INFO:tasks.cephadm:Adding mon.b on smithi134 2023-06-23T01:12:13.301 INFO:tasks.cephadm:Adding mon.c on smithi157 2023-06-23T01:12:13.302 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph orch apply mon '3;smithi120:172.21.15.120=a;smithi134:172.21.15.134=b;smithi157:172.21.15.157=c' 2023-06-23T01:12:13.913 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-23T01:12:14.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:13 smithi120 ceph-mon[79813]: Updating smithi157:/var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-23T01:12:14.299 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:13 smithi120 ceph-mon[79813]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2023-06-23T01:12:14.299 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:13 smithi120 ceph-mon[79813]: Updating smithi157:/var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/config/ceph.client.admin.keyring 2023-06-23T01:12:14.299 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:13 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/2579866356' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-06-23T01:12:14.299 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:13 smithi120 ceph-mon[79813]: osdmap e4: 0 total, 0 up, 0 in 2023-06-23T01:12:15.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:14 smithi120 ceph-mon[79813]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:15.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.410+0000 7efe96acf700 1 -- 172.21.15.157:0/1778638579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe90101620 msgr2=0x7efe901019f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:12:15.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.410+0000 7efe96acf700 1 --2- 172.21.15.157:0/1778638579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe90101620 0x7efe901019f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efe840048c0 tx=0x7efe84009e30 comp rx=0 tx=0).stop 2023-06-23T01:12:15.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.411+0000 7efe96acf700 1 -- 172.21.15.157:0/1778638579 shutdown_connections 2023-06-23T01:12:15.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.411+0000 7efe96acf700 1 --2- 172.21.15.157:0/1778638579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe90101620 0x7efe901019f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:15.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.411+0000 7efe96acf700 1 -- 172.21.15.157:0/1778638579 >> 172.21.15.157:0/1778638579 conn(0x7efe90071b50 msgr2=0x7efe90071f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:12:15.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.411+0000 7efe96acf700 1 -- 172.21.15.157:0/1778638579 shutdown_connections 2023-06-23T01:12:15.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.411+0000 7efe96acf700 1 -- 172.21.15.157:0/1778638579 wait complete. 2023-06-23T01:12:15.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.411+0000 7efe96acf700 1 Processor -- start 2023-06-23T01:12:15.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.411+0000 7efe96acf700 1 -- start start 2023-06-23T01:12:15.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.411+0000 7efe96acf700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe90101620 0x7efe90111800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:12:15.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.411+0000 7efe96acf700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe90111d40 con 0x7efe90101620 2023-06-23T01:12:15.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.412+0000 7efe9486b700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe90101620 0x7efe90111800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:12:15.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.412+0000 7efe9486b700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe90101620 0x7efe90111800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.157:40252/0 (socket says 172.21.15.157:40252) 2023-06-23T01:12:15.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.412+0000 7efe9486b700 1 -- 172.21.15.157:0/4290152296 learned_addr learned my addr 172.21.15.157:0/4290152296 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-06-23T01:12:15.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.412+0000 7efe9486b700 1 -- 172.21.15.157:0/4290152296 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe84005040 con 0x7efe90101620 2023-06-23T01:12:15.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.412+0000 7efe9486b700 1 --2- 172.21.15.157:0/4290152296 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe90101620 0x7efe90111800 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efe84008f00 tx=0x7efe84009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:12:15.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.412+0000 7efe8d7fa700 1 -- 172.21.15.157:0/4290152296 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe8400e040 con 0x7efe90101620 2023-06-23T01:12:15.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.412+0000 7efe96acf700 1 -- 172.21.15.157:0/4290152296 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe90111f40 con 0x7efe90101620 2023-06-23T01:12:15.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.412+0000 7efe8d7fa700 1 -- 172.21.15.157:0/4290152296 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe84009840 con 0x7efe90101620 2023-06-23T01:12:15.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.412+0000 7efe8d7fa700 1 -- 172.21.15.157:0/4290152296 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe84013e30 con 0x7efe90101620 2023-06-23T01:12:15.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.412+0000 7efe96acf700 1 -- 172.21.15.157:0/4290152296 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe9010b6c0 con 0x7efe90101620 2023-06-23T01:12:15.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.413+0000 7efe8d7fa700 1 -- 172.21.15.157:0/4290152296 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 13) v1 ==== 43276+0+0 (secure 0 0 0) 0x7efe84013520 con 0x7efe90101620 2023-06-23T01:12:15.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.413+0000 7efe96acf700 1 -- 172.21.15.157:0/4290152296 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe9005f060 con 0x7efe90101620 2023-06-23T01:12:15.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.413+0000 7efe8d7fa700 1 --2- 172.21.15.157:0/4290152296 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7efe80036660 0x7efe80038b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:12:15.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.413+0000 7efe8d7fa700 1 -- 172.21.15.157:0/4290152296 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7efe84049580 con 0x7efe90101620 2023-06-23T01:12:15.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.414+0000 7efe8ffff700 1 --2- 172.21.15.157:0/4290152296 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7efe80036660 0x7efe80038b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:12:15.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.414+0000 7efe8ffff700 1 --2- 172.21.15.157:0/4290152296 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7efe80036660 0x7efe80038b10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efe7c009990 tx=0x7efe7c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:12:15.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.419+0000 7efe8d7fa700 1 -- 172.21.15.157:0/4290152296 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7efe840117e0 con 0x7efe90101620 2023-06-23T01:12:15.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.613+0000 7efe96acf700 1 -- 172.21.15.157:0/4290152296 --> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi120:172.21.15.120=a;smithi134:172.21.15.134=b;smithi157:172.21.15.157=c", "target": ["mon-mgr", ""]}) v1 -- 0x7efe9010cde0 con 0x7efe80036660 2023-06-23T01:12:15.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.620+0000 7efe8d7fa700 1 -- 172.21.15.157:0/4290152296 <== mgr.14152 v2:172.21.15.120:6800/1770912857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7efe9010cde0 con 0x7efe80036660 2023-06-23T01:12:15.624 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled mon update... 2023-06-23T01:12:15.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.622+0000 7efe96acf700 1 -- 172.21.15.157:0/4290152296 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7efe80036660 msgr2=0x7efe80038b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:12:15.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.622+0000 7efe96acf700 1 --2- 172.21.15.157:0/4290152296 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7efe80036660 0x7efe80038b10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efe7c009990 tx=0x7efe7c006e40 comp rx=0 tx=0).stop 2023-06-23T01:12:15.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.622+0000 7efe96acf700 1 -- 172.21.15.157:0/4290152296 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe90101620 msgr2=0x7efe90111800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:12:15.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.622+0000 7efe96acf700 1 --2- 172.21.15.157:0/4290152296 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe90101620 0x7efe90111800 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efe84008f00 tx=0x7efe84009e30 comp rx=0 tx=0).stop 2023-06-23T01:12:15.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.622+0000 7efe96acf700 1 -- 172.21.15.157:0/4290152296 shutdown_connections 2023-06-23T01:12:15.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.622+0000 7efe96acf700 1 --2- 172.21.15.157:0/4290152296 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7efe80036660 0x7efe80038b10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:15.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.622+0000 7efe96acf700 1 --2- 172.21.15.157:0/4290152296 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe90101620 0x7efe90111800 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:15.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.622+0000 7efe96acf700 1 -- 172.21.15.157:0/4290152296 >> 172.21.15.157:0/4290152296 conn(0x7efe90071b50 msgr2=0x7efe90102bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:12:15.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.622+0000 7efe96acf700 1 -- 172.21.15.157:0/4290152296 shutdown_connections 2023-06-23T01:12:15.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:15.622+0000 7efe96acf700 1 -- 172.21.15.157:0/4290152296 wait complete. 2023-06-23T01:12:16.261 DEBUG:teuthology.orchestra.run.smithi134:mon.b> sudo journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mon.b.service 2023-06-23T01:12:16.263 DEBUG:teuthology.orchestra.run.smithi157:mon.c> sudo journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mon.c.service 2023-06-23T01:12:16.265 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-06-23T01:12:16.265 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph mon dump -f json 2023-06-23T01:12:16.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:-- Logs begin at Fri 2023-06-23 00:49:55 UTC. -- 2023-06-23T01:12:16.296 INFO:journalctl@ceph.mon.b.smithi134.stdout:-- Logs begin at Fri 2023-06-23 00:49:19 UTC. -- 2023-06-23T01:12:17.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:16 smithi120 ceph-mon[79813]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:17.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:16 smithi120 ceph-mon[79813]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi120:172.21.15.120=a;smithi134:172.21.15.134=b;smithi157:172.21.15.157=c", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:12:17.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:16 smithi120 ceph-mon[79813]: Saving service mon spec with placement smithi120:172.21.15.120=a;smithi134:172.21.15.134=b;smithi157:172.21.15.157=c;count:3 2023-06-23T01:12:17.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:16 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:17.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:16 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:12:17.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:16 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:17.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:16 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:12:17.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:16 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:17.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:16 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-23T01:12:17.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:16 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:17.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:16 smithi120 ceph-mon[79813]: Deploying daemon mon.c on smithi157 2023-06-23T01:12:17.731 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-23T01:12:19.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:18 smithi120 ceph-mon[79813]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:19.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:19.979+0000 7ff9c88ed700 1 -- 172.21.15.157:0/1057629676 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9c41026c0 msgr2=0x7ff9c4102a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:12:19.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:19.979+0000 7ff9c88ed700 1 --2- 172.21.15.157:0/1057629676 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9c41026c0 0x7ff9c4102a90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff9b40048c0 tx=0x7ff9b4009e30 comp rx=0 tx=0).stop 2023-06-23T01:12:19.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:19.979+0000 7ff9c88ed700 1 -- 172.21.15.157:0/1057629676 shutdown_connections 2023-06-23T01:12:19.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:19.979+0000 7ff9c88ed700 1 --2- 172.21.15.157:0/1057629676 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9c41026c0 0x7ff9c4102a90 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:19.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:19.979+0000 7ff9c88ed700 1 -- 172.21.15.157:0/1057629676 >> 172.21.15.157:0/1057629676 conn(0x7ff9c40f8200 msgr2=0x7ff9c40fa610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:12:19.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:19.979+0000 7ff9c88ed700 1 -- 172.21.15.157:0/1057629676 shutdown_connections 2023-06-23T01:12:19.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:19.979+0000 7ff9c88ed700 1 -- 172.21.15.157:0/1057629676 wait complete. 2023-06-23T01:12:19.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:19.980+0000 7ff9c88ed700 1 Processor -- start 2023-06-23T01:12:19.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:19.980+0000 7ff9c88ed700 1 -- start start 2023-06-23T01:12:19.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:19.980+0000 7ff9c88ed700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9c41026c0 0x7ff9c41129a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:12:19.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:19.980+0000 7ff9c88ed700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c4112ee0 con 0x7ff9c41026c0 2023-06-23T01:12:19.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:19.980+0000 7ff9c259c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9c41026c0 0x7ff9c41129a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:12:19.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:19.980+0000 7ff9c259c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9c41026c0 0x7ff9c41129a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.157:40258/0 (socket says 172.21.15.157:40258) 2023-06-23T01:12:19.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:19.980+0000 7ff9c259c700 1 -- 172.21.15.157:0/3261027134 learned_addr learned my addr 172.21.15.157:0/3261027134 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-06-23T01:12:19.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:19.981+0000 7ff9c259c700 1 -- 172.21.15.157:0/3261027134 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b4005040 con 0x7ff9c41026c0 2023-06-23T01:12:19.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:19.981+0000 7ff9c259c700 1 --2- 172.21.15.157:0/3261027134 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9c41026c0 0x7ff9c41129a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff9b400b600 tx=0x7ff9b4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:12:19.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:19.981+0000 7ff9b2ffd700 1 -- 172.21.15.157:0/3261027134 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9b400e040 con 0x7ff9c41026c0 2023-06-23T01:12:19.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:19.981+0000 7ff9c88ed700 1 -- 172.21.15.157:0/3261027134 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9c41130e0 con 0x7ff9c41026c0 2023-06-23T01:12:19.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:19.981+0000 7ff9b2ffd700 1 -- 172.21.15.157:0/3261027134 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9b40098c0 con 0x7ff9c41026c0 2023-06-23T01:12:19.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:19.981+0000 7ff9b2ffd700 1 -- 172.21.15.157:0/3261027134 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9b4013de0 con 0x7ff9c41026c0 2023-06-23T01:12:19.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:19.981+0000 7ff9c88ed700 1 -- 172.21.15.157:0/3261027134 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9c4113550 con 0x7ff9c41026c0 2023-06-23T01:12:19.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:19.982+0000 7ff9c88ed700 1 -- 172.21.15.157:0/3261027134 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9c405f060 con 0x7ff9c41026c0 2023-06-23T01:12:19.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:19.982+0000 7ff9b2ffd700 1 -- 172.21.15.157:0/3261027134 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 13) v1 ==== 43276+0+0 (secure 0 0 0) 0x7ff9b40135b0 con 0x7ff9c41026c0 2023-06-23T01:12:19.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:19.982+0000 7ff9b2ffd700 1 --2- 172.21.15.157:0/3261027134 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7ff9ac036660 0x7ff9ac038b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:12:19.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:19.982+0000 7ff9b2ffd700 1 -- 172.21.15.157:0/3261027134 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7ff9b4049500 con 0x7ff9c41026c0 2023-06-23T01:12:19.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:19.983+0000 7ff9c1d9b700 1 --2- 172.21.15.157:0/3261027134 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7ff9ac036660 0x7ff9ac038b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:12:19.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:19.983+0000 7ff9c1d9b700 1 --2- 172.21.15.157:0/3261027134 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7ff9ac036660 0x7ff9ac038b10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8009990 tx=0x7ff9b8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:12:19.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:19.988+0000 7ff9b2ffd700 1 -- 172.21.15.157:0/3261027134 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7ff9b4011bf0 con 0x7ff9c41026c0 2023-06-23T01:12:20.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:20.050+0000 7ff9b2ffd700 1 -- 172.21.15.157:0/3261027134 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7ff9b4013420 con 0x7ff9c41026c0 2023-06-23T01:12:20.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:20.257+0000 7ff9c88ed700 1 -- 172.21.15.157:0/3261027134 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff9c404b970 con 0x7ff9c41026c0 2023-06-23T01:12:20.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:20 smithi157 ceph-mon[116945]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-06-23T01:12:20.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:20 smithi157 systemd[1]: Started Ceph mon.c for 9efbfa8e-1162-11ee-9b2b-001a4aab830c. 2023-06-23T01:12:25.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:25.073+0000 7ff9b2ffd700 1 -- 172.21.15.157:0/3261027134 <== mon.0 v2:172.21.15.120:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1005 (secure 0 0 0) 0x7ff9b401be90 con 0x7ff9c41026c0 2023-06-23T01:12:25.077 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-06-23T01:12:25.077 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":2,"fsid":"9efbfa8e-1162-11ee-9b2b-001a4aab830c","modified":"2023-06-23T01:12:20.049975Z","created":"2023-06-23T01:10:35.325531Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:3300","nonce":0},{"type":"v1","addr":"172.21.15.120:6789","nonce":0}]},"addr":"172.21.15.120:6789/0","public_addr":"172.21.15.120:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:3300","nonce":0},{"type":"v1","addr":"172.21.15.157:6789","nonce":0}]},"addr":"172.21.15.157:6789/0","public_addr":"172.21.15.157:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-06-23T01:12:25.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:25.076+0000 7ff9c88ed700 1 -- 172.21.15.157:0/3261027134 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7ff9ac036660 msgr2=0x7ff9ac038b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:12:25.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:25.076+0000 7ff9c88ed700 1 --2- 172.21.15.157:0/3261027134 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7ff9ac036660 0x7ff9ac038b10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8009990 tx=0x7ff9b8006e40 comp rx=0 tx=0).stop 2023-06-23T01:12:25.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:25.076+0000 7ff9c88ed700 1 -- 172.21.15.157:0/3261027134 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9c41026c0 msgr2=0x7ff9c41129a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:12:25.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:25.076+0000 7ff9c88ed700 1 --2- 172.21.15.157:0/3261027134 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9c41026c0 0x7ff9c41129a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff9b400b600 tx=0x7ff9b4009e30 comp rx=0 tx=0).stop 2023-06-23T01:12:25.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:25.076+0000 7ff9c88ed700 1 -- 172.21.15.157:0/3261027134 shutdown_connections 2023-06-23T01:12:25.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:25.076+0000 7ff9c88ed700 1 --2- 172.21.15.157:0/3261027134 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7ff9ac036660 0x7ff9ac038b10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:25.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:25.076+0000 7ff9c88ed700 1 --2- 172.21.15.157:0/3261027134 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9c41026c0 0x7ff9c41129a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:25.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:25.076+0000 7ff9c88ed700 1 -- 172.21.15.157:0/3261027134 >> 172.21.15.157:0/3261027134 conn(0x7ff9c40f8200 msgr2=0x7ff9c40f8e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:12:25.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:25.076+0000 7ff9c88ed700 1 -- 172.21.15.157:0/3261027134 shutdown_connections 2023-06-23T01:12:25.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:25.076+0000 7ff9c88ed700 1 -- 172.21.15.157:0/3261027134 wait complete. 2023-06-23T01:12:25.083 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 2 2023-06-23T01:12:25.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:25 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-23T01:12:25.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:25 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T01:12:25.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:25 smithi157 ceph-mon[116945]: mon.a calling monitor election 2023-06-23T01:12:25.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:25 smithi157 ceph-mon[116945]: from='client.? 172.21.15.157:0/3261027134' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-23T01:12:25.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:25 smithi157 ceph-mon[116945]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:25.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:25 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T01:12:25.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:25 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T01:12:25.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:25 smithi157 ceph-mon[116945]: mon.c calling monitor election 2023-06-23T01:12:25.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:25 smithi157 ceph-mon[116945]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:25.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:25 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T01:12:25.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:25 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T01:12:25.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:25 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T01:12:25.379 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:25 smithi157 ceph-mon[116945]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-06-23T01:12:25.379 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:25 smithi157 ceph-mon[116945]: monmap e2: 2 mons at {a=[v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0],c=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} 2023-06-23T01:12:25.379 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:25 smithi157 ceph-mon[116945]: fsmap 2023-06-23T01:12:25.379 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:25 smithi157 ceph-mon[116945]: osdmap e4: 0 total, 0 up, 0 in 2023-06-23T01:12:25.380 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:25 smithi157 ceph-mon[116945]: mgrmap e13: x(active, since 46s) 2023-06-23T01:12:25.380 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:25 smithi157 ceph-mon[116945]: overall HEALTH_OK 2023-06-23T01:12:25.380 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:25 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:25.381 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:25 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:25.381 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:25 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:25.381 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:25 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-23T01:12:25.381 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:25 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:25.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:25 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-23T01:12:25.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:25 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T01:12:25.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:25 smithi120 ceph-mon[79813]: mon.a calling monitor election 2023-06-23T01:12:25.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:25 smithi120 ceph-mon[79813]: from='client.? 172.21.15.157:0/3261027134' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-23T01:12:25.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:25 smithi120 ceph-mon[79813]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:25.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:25 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T01:12:25.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:25 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T01:12:25.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:25 smithi120 ceph-mon[79813]: mon.c calling monitor election 2023-06-23T01:12:25.512 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:25 smithi120 ceph-mon[79813]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:25.512 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:25 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T01:12:25.512 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:25 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T01:12:25.512 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:25 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T01:12:25.512 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:25 smithi120 ceph-mon[79813]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-06-23T01:12:25.512 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:25 smithi120 ceph-mon[79813]: monmap e2: 2 mons at {a=[v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0],c=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} 2023-06-23T01:12:25.513 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:25 smithi120 ceph-mon[79813]: fsmap 2023-06-23T01:12:25.513 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:25 smithi120 ceph-mon[79813]: osdmap e4: 0 total, 0 up, 0 in 2023-06-23T01:12:25.513 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:25 smithi120 ceph-mon[79813]: mgrmap e13: x(active, since 46s) 2023-06-23T01:12:25.513 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:25 smithi120 ceph-mon[79813]: overall HEALTH_OK 2023-06-23T01:12:25.513 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:25 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:25.513 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:25 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:25.513 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:25 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:25.513 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:25 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-23T01:12:25.514 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:25 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:26.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:26 smithi120 ceph-mon[79813]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:26.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:26 smithi120 ceph-mon[79813]: Deploying daemon mon.b on smithi134 2023-06-23T01:12:26.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:26 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T01:12:26.626 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:26 smithi157 ceph-mon[116945]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:26.626 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:26 smithi157 ceph-mon[116945]: Deploying daemon mon.b on smithi134 2023-06-23T01:12:26.627 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:26 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T01:12:26.900 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-06-23T01:12:26.901 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph mon dump -f json 2023-06-23T01:12:27.491 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.c/config 2023-06-23T01:12:28.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:27 smithi134 systemd[1]: Starting Ceph mon.b for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:12:28.608 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: set uid:gid to 167:167 (ceph:ceph) 2023-06-23T01:12:28.608 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: ceph version 17.2.6-719-g42ddc46e (42ddc46e6e6582d8e29a5a982a81886fe028be2c) quincy (stable), process ceph-mon, pid 7 2023-06-23T01:12:28.608 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: pidfile_write: ignore empty --pid-file 2023-06-23T01:12:28.608 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: load: jerasure load: lrc 2023-06-23T01:12:28.608 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: RocksDB version: 6.15.5 2023-06-23T01:12:28.609 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2023-06-23T01:12:28.609 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Compile date Jun 21 2023 2023-06-23T01:12:28.609 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: DB SUMMARY 2023-06-23T01:12:28.609 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: DB Session ID: BAR7O31GKQJIXAD34U15 2023-06-23T01:12:28.609 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: CURRENT file: CURRENT 2023-06-23T01:12:28.609 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: IDENTITY file: IDENTITY 2023-06-23T01:12:28.609 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2023-06-23T01:12:28.609 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-06-23T01:12:28.610 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-06-23T01:12:28.610 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.error_if_exists: 0 2023-06-23T01:12:28.610 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.create_if_missing: 0 2023-06-23T01:12:28.610 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.paranoid_checks: 1 2023-06-23T01:12:28.610 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-06-23T01:12:28.610 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.env: 0x55b74948ca80 2023-06-23T01:12:28.610 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.fs: Posix File System 2023-06-23T01:12:28.610 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.info_log: 0x55b74a5b1da0 2023-06-23T01:12:28.611 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.max_file_opening_threads: 16 2023-06-23T01:12:28.611 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.statistics: (nil) 2023-06-23T01:12:28.611 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.use_fsync: 0 2023-06-23T01:12:28.611 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.max_log_file_size: 0 2023-06-23T01:12:28.611 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-06-23T01:12:28.611 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.log_file_time_to_roll: 0 2023-06-23T01:12:28.611 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.keep_log_file_num: 1000 2023-06-23T01:12:28.611 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.recycle_log_file_num: 0 2023-06-23T01:12:28.612 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.allow_fallocate: 1 2023-06-23T01:12:28.612 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.allow_mmap_reads: 0 2023-06-23T01:12:28.612 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.allow_mmap_writes: 0 2023-06-23T01:12:28.612 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.use_direct_reads: 0 2023-06-23T01:12:28.612 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-06-23T01:12:28.612 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.create_missing_column_families: 0 2023-06-23T01:12:28.612 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.db_log_dir: 2023-06-23T01:12:28.613 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-b/store.db 2023-06-23T01:12:28.613 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.table_cache_numshardbits: 6 2023-06-23T01:12:28.613 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.WAL_ttl_seconds: 0 2023-06-23T01:12:28.613 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.WAL_size_limit_MB: 0 2023-06-23T01:12:28.613 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-06-23T01:12:28.613 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-06-23T01:12:28.613 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.is_fd_close_on_exec: 1 2023-06-23T01:12:28.613 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.advise_random_on_open: 1 2023-06-23T01:12:28.614 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.db_write_buffer_size: 0 2023-06-23T01:12:28.614 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.write_buffer_manager: 0x55b74a6788d0 2023-06-23T01:12:28.614 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-06-23T01:12:28.614 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2023-06-23T01:12:28.614 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-06-23T01:12:28.614 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.use_adaptive_mutex: 0 2023-06-23T01:12:28.614 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.rate_limiter: (nil) 2023-06-23T01:12:28.614 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-06-23T01:12:28.615 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.wal_recovery_mode: 2 2023-06-23T01:12:28.615 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.enable_thread_tracking: 0 2023-06-23T01:12:28.615 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.enable_pipelined_write: 0 2023-06-23T01:12:28.615 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.unordered_write: 0 2023-06-23T01:12:28.615 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-06-23T01:12:28.615 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-06-23T01:12:28.615 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-06-23T01:12:28.615 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-06-23T01:12:28.616 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.row_cache: None 2023-06-23T01:12:28.616 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.wal_filter: None 2023-06-23T01:12:28.616 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-06-23T01:12:28.616 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.allow_ingest_behind: 0 2023-06-23T01:12:28.616 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.preserve_deletes: 0 2023-06-23T01:12:28.616 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.two_write_queues: 0 2023-06-23T01:12:28.617 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.manual_wal_flush: 0 2023-06-23T01:12:28.617 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.atomic_flush: 0 2023-06-23T01:12:28.617 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-06-23T01:12:28.617 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.persist_stats_to_disk: 0 2023-06-23T01:12:28.617 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.write_dbid_to_manifest: 0 2023-06-23T01:12:28.617 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.log_readahead_size: 0 2023-06-23T01:12:28.617 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-06-23T01:12:28.617 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.best_efforts_recovery: 0 2023-06-23T01:12:28.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-06-23T01:12:28.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-06-23T01:12:28.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.allow_data_in_errors: 0 2023-06-23T01:12:28.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.db_host_id: __hostname__ 2023-06-23T01:12:28.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.max_background_jobs: 2 2023-06-23T01:12:28.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.max_background_compactions: -1 2023-06-23T01:12:28.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.max_subcompactions: 1 2023-06-23T01:12:28.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-06-23T01:12:28.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-06-23T01:12:28.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.delayed_write_rate : 16777216 2023-06-23T01:12:28.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.max_total_wal_size: 0 2023-06-23T01:12:28.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-06-23T01:12:28.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.stats_dump_period_sec: 600 2023-06-23T01:12:28.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.stats_persist_period_sec: 600 2023-06-23T01:12:28.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-06-23T01:12:28.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.max_open_files: -1 2023-06-23T01:12:28.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.bytes_per_sync: 0 2023-06-23T01:12:28.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.wal_bytes_per_sync: 0 2023-06-23T01:12:28.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.strict_bytes_per_sync: 0 2023-06-23T01:12:28.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.compaction_readahead_size: 0 2023-06-23T01:12:28.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.max_background_flushes: -1 2023-06-23T01:12:28.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Compression algorithms supported: 2023-06-23T01:12:28.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-06-23T01:12:28.621 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: kZSTD supported: 0 2023-06-23T01:12:28.621 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: kXpressCompression supported: 0 2023-06-23T01:12:28.621 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: kLZ4HCCompression supported: 1 2023-06-23T01:12:28.621 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: kZlibCompression supported: 1 2023-06-23T01:12:28.621 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: kSnappyCompression supported: 1 2023-06-23T01:12:28.621 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: kLZ4Compression supported: 1 2023-06-23T01:12:28.621 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: kBZip2Compression supported: 0 2023-06-23T01:12:28.621 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Fast CRC32 supported: Supported on x86 2023-06-23T01:12:28.622 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000003 2023-06-23T01:12:28.622 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2023-06-23T01:12:28.622 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-06-23T01:12:28.622 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.merge_operator: 2023-06-23T01:12:28.622 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.compaction_filter: None 2023-06-23T01:12:28.622 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.compaction_filter_factory: None 2023-06-23T01:12:28.622 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.sst_partitioner_factory: None 2023-06-23T01:12:28.622 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.memtable_factory: SkipListFactory 2023-06-23T01:12:28.623 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.table_factory: BlockBasedTable 2023-06-23T01:12:28.623 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b74a57cd30) 2023-06-23T01:12:28.623 INFO:journalctl@ceph.mon.b.smithi134.stdout: cache_index_and_filter_blocks: 1 2023-06-23T01:12:28.623 INFO:journalctl@ceph.mon.b.smithi134.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-06-23T01:12:28.623 INFO:journalctl@ceph.mon.b.smithi134.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-06-23T01:12:28.623 INFO:journalctl@ceph.mon.b.smithi134.stdout: pin_top_level_index_and_filter: 1 2023-06-23T01:12:28.624 INFO:journalctl@ceph.mon.b.smithi134.stdout: index_type: 0 2023-06-23T01:12:28.625 INFO:journalctl@ceph.mon.b.smithi134.stdout: data_block_index_type: 0 2023-06-23T01:12:28.625 INFO:journalctl@ceph.mon.b.smithi134.stdout: index_shortening: 1 2023-06-23T01:12:28.625 INFO:journalctl@ceph.mon.b.smithi134.stdout: data_block_hash_table_util_ratio: 0.750000 2023-06-23T01:12:28.625 INFO:journalctl@ceph.mon.b.smithi134.stdout: hash_index_allow_collision: 1 2023-06-23T01:12:28.625 INFO:journalctl@ceph.mon.b.smithi134.stdout: checksum: 1 2023-06-23T01:12:28.625 INFO:journalctl@ceph.mon.b.smithi134.stdout: no_block_cache: 0 2023-06-23T01:12:28.626 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_cache: 0x55b74a63b1f0 2023-06-23T01:12:28.626 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_cache_name: BinnedLRUCache 2023-06-23T01:12:28.626 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_cache_options: 2023-06-23T01:12:28.626 INFO:journalctl@ceph.mon.b.smithi134.stdout: capacity : 536870912 2023-06-23T01:12:28.626 INFO:journalctl@ceph.mon.b.smithi134.stdout: num_shard_bits : 4 2023-06-23T01:12:28.626 INFO:journalctl@ceph.mon.b.smithi134.stdout: strict_capacity_limit : 0 2023-06-23T01:12:28.626 INFO:journalctl@ceph.mon.b.smithi134.stdout: high_pri_pool_ratio: 0.000 2023-06-23T01:12:28.626 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_cache_compressed: (nil) 2023-06-23T01:12:28.627 INFO:journalctl@ceph.mon.b.smithi134.stdout: persistent_cache: (nil) 2023-06-23T01:12:28.627 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_size: 4096 2023-06-23T01:12:28.627 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_size_deviation: 10 2023-06-23T01:12:28.627 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_restart_interval: 16 2023-06-23T01:12:28.627 INFO:journalctl@ceph.mon.b.smithi134.stdout: index_block_restart_interval: 1 2023-06-23T01:12:28.627 INFO:journalctl@ceph.mon.b.smithi134.stdout: metadata_block_size: 4096 2023-06-23T01:12:28.627 INFO:journalctl@ceph.mon.b.smithi134.stdout: partition_filters: 0 2023-06-23T01:12:28.627 INFO:journalctl@ceph.mon.b.smithi134.stdout: use_delta_encoding: 1 2023-06-23T01:12:28.628 INFO:journalctl@ceph.mon.b.smithi134.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2023-06-23T01:12:28.628 INFO:journalctl@ceph.mon.b.smithi134.stdout: whole_key_filtering: 1 2023-06-23T01:12:28.628 INFO:journalctl@ceph.mon.b.smithi134.stdout: verify_compression: 0 2023-06-23T01:12:28.628 INFO:journalctl@ceph.mon.b.smithi134.stdout: read_amp_bytes_per_bit: 0 2023-06-23T01:12:28.628 INFO:journalctl@ceph.mon.b.smithi134.stdout: format_version: 4 2023-06-23T01:12:28.628 INFO:journalctl@ceph.mon.b.smithi134.stdout: enable_index_compression: 1 2023-06-23T01:12:28.628 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_align: 0 2023-06-23T01:12:28.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.write_buffer_size: 33554432 2023-06-23T01:12:28.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.max_write_buffer_number: 2 2023-06-23T01:12:28.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.compression: NoCompression 2023-06-23T01:12:28.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.bottommost_compression: Disabled 2023-06-23T01:12:28.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.prefix_extractor: nullptr 2023-06-23T01:12:28.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-06-23T01:12:28.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.num_levels: 7 2023-06-23T01:12:28.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-06-23T01:12:28.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-06-23T01:12:28.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-06-23T01:12:28.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-06-23T01:12:28.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-06-23T01:12:28.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-06-23T01:12:28.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-06-23T01:12:28.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-06-23T01:12:28.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-06-23T01:12:28.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-06-23T01:12:28.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.compression_opts.window_bits: -14 2023-06-23T01:12:28.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.compression_opts.level: 32767 2023-06-23T01:12:28.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.compression_opts.strategy: 0 2023-06-23T01:12:28.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-06-23T01:12:28.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-06-23T01:12:28.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-06-23T01:12:28.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.compression_opts.enabled: false 2023-06-23T01:12:28.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-06-23T01:12:28.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-06-23T01:12:28.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-06-23T01:12:28.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.target_file_size_base: 67108864 2023-06-23T01:12:28.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.target_file_size_multiplier: 1 2023-06-23T01:12:28.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-06-23T01:12:28.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-06-23T01:12:28.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-06-23T01:12:28.633 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-06-23T01:12:28.633 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-06-23T01:12:28.633 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-06-23T01:12:28.633 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-06-23T01:12:28.633 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-06-23T01:12:28.633 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-06-23T01:12:28.633 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-06-23T01:12:28.634 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-06-23T01:12:28.634 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-06-23T01:12:28.634 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.arena_block_size: 4194304 2023-06-23T01:12:28.634 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-06-23T01:12:28.634 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-06-23T01:12:28.634 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2023-06-23T01:12:28.634 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.disable_auto_compactions: 0 2023-06-23T01:12:28.634 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-06-23T01:12:28.635 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-06-23T01:12:28.635 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-06-23T01:12:28.635 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-06-23T01:12:28.635 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-06-23T01:12:28.635 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-06-23T01:12:28.635 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-06-23T01:12:28.635 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-06-23T01:12:28.636 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-06-23T01:12:28.636 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-06-23T01:12:28.636 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.table_properties_collectors: 2023-06-23T01:12:28.636 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.inplace_update_support: 0 2023-06-23T01:12:28.636 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.inplace_update_num_locks: 10000 2023-06-23T01:12:28.636 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-06-23T01:12:28.636 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-06-23T01:12:28.637 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.memtable_huge_page_size: 0 2023-06-23T01:12:28.637 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.bloom_locality: 0 2023-06-23T01:12:28.637 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.max_successive_merges: 0 2023-06-23T01:12:28.637 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.optimize_filters_for_hits: 0 2023-06-23T01:12:28.637 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.paranoid_file_checks: 0 2023-06-23T01:12:28.637 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.force_consistency_checks: 1 2023-06-23T01:12:28.637 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.report_bg_io_stats: 0 2023-06-23T01:12:28.637 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.ttl: 2592000 2023-06-23T01:12:28.638 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.periodic_compaction_seconds: 0 2023-06-23T01:12:28.638 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.enable_blob_files: false 2023-06-23T01:12:28.638 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.min_blob_size: 0 2023-06-23T01:12:28.638 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.blob_file_size: 268435456 2023-06-23T01:12:28.638 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.blob_compression_type: NoCompression 2023-06-23T01:12:28.638 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.enable_blob_garbage_collection: false 2023-06-23T01:12:28.638 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-06-23T01:12:28.638 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-06-23T01:12:28.639 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2023-06-23T01:12:28.639 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2023-06-23T01:12:28.639 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: EVENT_LOG_v1 {"time_micros": 1687482748514038, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-06-23T01:12:28.639 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2023-06-23T01:12:28.639 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2023-06-23T01:12:28.639 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: EVENT_LOG_v1 {"time_micros": 1687482748514316, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1670, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1687482748, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "7c702c47-1bef-4364-9eb2-0c495357055d", "db_session_id": "BAR7O31GKQJIXAD34U15"}} 2023-06-23T01:12:28.639 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2023-06-23T01:12:28.640 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: EVENT_LOG_v1 {"time_micros": 1687482748514617, "job": 1, "event": "recovery_finished"} 2023-06-23T01:12:28.640 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-06-23T01:12:28.640 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x55b74a5cf180 2023-06-23T01:12:28.640 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: DB pointer 0x55b74a5ea000 2023-06-23T01:12:28.640 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2023-06-23T01:12:28.640 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: rocksdb: [db/db_impl/db_impl.cc:903] 2023-06-23T01:12:28.640 INFO:journalctl@ceph.mon.b.smithi134.stdout: ** DB Stats ** 2023-06-23T01:12:28.641 INFO:journalctl@ceph.mon.b.smithi134.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-06-23T01:12:28.641 INFO:journalctl@ceph.mon.b.smithi134.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-06-23T01:12:28.641 INFO:journalctl@ceph.mon.b.smithi134.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-06-23T01:12:28.641 INFO:journalctl@ceph.mon.b.smithi134.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-06-23T01:12:28.641 INFO:journalctl@ceph.mon.b.smithi134.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-06-23T01:12:28.641 INFO:journalctl@ceph.mon.b.smithi134.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2023-06-23T01:12:28.641 INFO:journalctl@ceph.mon.b.smithi134.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-06-23T01:12:28.641 INFO:journalctl@ceph.mon.b.smithi134.stdout: 2023-06-23T01:12:28.642 INFO:journalctl@ceph.mon.b.smithi134.stdout: ** Compaction Stats [default] ** 2023-06-23T01:12:28.642 INFO:journalctl@ceph.mon.b.smithi134.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-06-23T01:12:28.642 INFO:journalctl@ceph.mon.b.smithi134.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-06-23T01:12:28.642 INFO:journalctl@ceph.mon.b.smithi134.stdout: L0 1/0 1.63 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 6.1 0.00 0.00 1 0.000 0 0 2023-06-23T01:12:28.642 INFO:journalctl@ceph.mon.b.smithi134.stdout: Sum 1/0 1.63 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 6.1 0.00 0.00 1 0.000 0 0 2023-06-23T01:12:28.642 INFO:journalctl@ceph.mon.b.smithi134.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 6.1 0.00 0.00 1 0.000 0 0 2023-06-23T01:12:28.642 INFO:journalctl@ceph.mon.b.smithi134.stdout: 2023-06-23T01:12:28.642 INFO:journalctl@ceph.mon.b.smithi134.stdout: ** Compaction Stats [default] ** 2023-06-23T01:12:28.643 INFO:journalctl@ceph.mon.b.smithi134.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-06-23T01:12:28.643 INFO:journalctl@ceph.mon.b.smithi134.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-06-23T01:12:28.643 INFO:journalctl@ceph.mon.b.smithi134.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 6.1 0.00 0.00 1 0.000 0 0 2023-06-23T01:12:28.643 INFO:journalctl@ceph.mon.b.smithi134.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-06-23T01:12:28.643 INFO:journalctl@ceph.mon.b.smithi134.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-06-23T01:12:28.643 INFO:journalctl@ceph.mon.b.smithi134.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-06-23T01:12:28.643 INFO:journalctl@ceph.mon.b.smithi134.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-06-23T01:12:28.644 INFO:journalctl@ceph.mon.b.smithi134.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-06-23T01:12:28.644 INFO:journalctl@ceph.mon.b.smithi134.stdout: AddFile(Keys): cumulative 0, interval 0 2023-06-23T01:12:28.644 INFO:journalctl@ceph.mon.b.smithi134.stdout: Cumulative compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-06-23T01:12:28.644 INFO:journalctl@ceph.mon.b.smithi134.stdout: Interval compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-06-23T01:12:28.644 INFO:journalctl@ceph.mon.b.smithi134.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-06-23T01:12:28.644 INFO:journalctl@ceph.mon.b.smithi134.stdout: 2023-06-23T01:12:28.644 INFO:journalctl@ceph.mon.b.smithi134.stdout: ** File Read Latency Histogram By Level [default] ** 2023-06-23T01:12:28.645 INFO:journalctl@ceph.mon.b.smithi134.stdout: 2023-06-23T01:12:28.645 INFO:journalctl@ceph.mon.b.smithi134.stdout: ** Compaction Stats [default] ** 2023-06-23T01:12:28.645 INFO:journalctl@ceph.mon.b.smithi134.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-06-23T01:12:28.645 INFO:journalctl@ceph.mon.b.smithi134.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-06-23T01:12:28.645 INFO:journalctl@ceph.mon.b.smithi134.stdout: L0 1/0 1.63 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 6.1 0.00 0.00 1 0.000 0 0 2023-06-23T01:12:28.645 INFO:journalctl@ceph.mon.b.smithi134.stdout: Sum 1/0 1.63 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 6.1 0.00 0.00 1 0.000 0 0 2023-06-23T01:12:28.645 INFO:journalctl@ceph.mon.b.smithi134.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2023-06-23T01:12:28.646 INFO:journalctl@ceph.mon.b.smithi134.stdout: 2023-06-23T01:12:28.646 INFO:journalctl@ceph.mon.b.smithi134.stdout: ** Compaction Stats [default] ** 2023-06-23T01:12:28.646 INFO:journalctl@ceph.mon.b.smithi134.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-06-23T01:12:28.646 INFO:journalctl@ceph.mon.b.smithi134.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-06-23T01:12:28.646 INFO:journalctl@ceph.mon.b.smithi134.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 6.1 0.00 0.00 1 0.000 0 0 2023-06-23T01:12:28.646 INFO:journalctl@ceph.mon.b.smithi134.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-06-23T01:12:28.646 INFO:journalctl@ceph.mon.b.smithi134.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-06-23T01:12:28.646 INFO:journalctl@ceph.mon.b.smithi134.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-06-23T01:12:28.647 INFO:journalctl@ceph.mon.b.smithi134.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-06-23T01:12:28.647 INFO:journalctl@ceph.mon.b.smithi134.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-06-23T01:12:28.647 INFO:journalctl@ceph.mon.b.smithi134.stdout: AddFile(Keys): cumulative 0, interval 0 2023-06-23T01:12:28.647 INFO:journalctl@ceph.mon.b.smithi134.stdout: Cumulative compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-06-23T01:12:28.647 INFO:journalctl@ceph.mon.b.smithi134.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-06-23T01:12:28.647 INFO:journalctl@ceph.mon.b.smithi134.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-06-23T01:12:28.647 INFO:journalctl@ceph.mon.b.smithi134.stdout: 2023-06-23T01:12:28.647 INFO:journalctl@ceph.mon.b.smithi134.stdout: ** File Read Latency Histogram By Level [default] ** 2023-06-23T01:12:28.648 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-06-23T01:12:28.648 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: using public_addr v2:172.21.15.134:0/0 -> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2023-06-23T01:12:28.648 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: starting mon.b rank -1 at public addrs [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] at bind addrs [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c 2023-06-23T01:12:28.648 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: mon.b@-1(???) e0 preinit fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c 2023-06-23T01:12:28.648 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 bash[116081]: 787655058ed9f3737dc03b81dafc9af74f74588412911a9a63e7176b1323d1a5 2023-06-23T01:12:28.648 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: mon.b@-1(synchronizing).mds e1 new map 2023-06-23T01:12:28.648 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: mon.b@-1(synchronizing).mds e1 print_map 2023-06-23T01:12:28.649 INFO:journalctl@ceph.mon.b.smithi134.stdout: e1 2023-06-23T01:12:28.649 INFO:journalctl@ceph.mon.b.smithi134.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-06-23T01:12:28.649 INFO:journalctl@ceph.mon.b.smithi134.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-06-23T01:12:28.649 INFO:journalctl@ceph.mon.b.smithi134.stdout: legacy client fscid: -1 2023-06-23T01:12:28.649 INFO:journalctl@ceph.mon.b.smithi134.stdout: 2023-06-23T01:12:28.649 INFO:journalctl@ceph.mon.b.smithi134.stdout: No filesystems configured 2023-06-23T01:12:28.649 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-06-23T01:12:28.649 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-06-23T01:12:28.650 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-06-23T01:12:28.650 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-06-23T01:12:28.650 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-06-23T01:12:28.650 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-06-23T01:12:28.650 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-06-23T01:12:28.650 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-06-23T01:12:28.650 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-06-23T01:12:28.650 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-06-23T01:12:28.651 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:28.651 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:28.651 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:28.651 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:28.651 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:28.651 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi134", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:12:28.651 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:28.651 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:12:28.652 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: Updating smithi134:/etc/ceph/ceph.conf 2023-06-23T01:12:28.652 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: Updating smithi134:/var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-23T01:12:28.652 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi157", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:12:28.652 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2023-06-23T01:12:28.652 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: Updating smithi134:/var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/config/ceph.client.admin.keyring 2023-06-23T01:12:28.652 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: Deploying cephadm binary to smithi157 2023-06-23T01:12:28.652 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:28.652 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:28.653 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:28.653 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:28.653 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:28.653 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: Added host smithi157 2023-06-23T01:12:28.653 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:12:28.653 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:28.653 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:28.653 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:28.654 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:28.654 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:28.654 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-23T01:12:28.654 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:28.654 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:28.654 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:28.654 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:28.655 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:28.655 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi157", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:12:28.655 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:28.655 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:12:28.655 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: Updating smithi157:/etc/ceph/ceph.conf 2023-06-23T01:12:28.655 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/2579866356' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-06-23T01:12:28.655 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:28.655 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:28.656 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:28.656 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: Updating smithi157:/var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-23T01:12:28.656 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2023-06-23T01:12:28.656 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: Updating smithi157:/var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/config/ceph.client.admin.keyring 2023-06-23T01:12:28.656 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/2579866356' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-06-23T01:12:28.656 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: osdmap e4: 0 total, 0 up, 0 in 2023-06-23T01:12:28.656 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:28.657 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:28.657 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi120:172.21.15.120=a;smithi134:172.21.15.134=b;smithi157:172.21.15.157=c", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:12:28.657 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: Saving service mon spec with placement smithi120:172.21.15.120=a;smithi134:172.21.15.134=b;smithi157:172.21.15.157=c;count:3 2023-06-23T01:12:28.657 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:28.657 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:12:28.657 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:28.657 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:12:28.657 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:28.658 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-23T01:12:28.658 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:28.658 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: Deploying daemon mon.c on smithi157 2023-06-23T01:12:28.658 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:28.658 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-23T01:12:28.658 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T01:12:28.658 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: mon.a calling monitor election 2023-06-23T01:12:28.658 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='client.? 172.21.15.157:0/3261027134' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-23T01:12:28.659 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:28.659 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T01:12:28.659 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T01:12:28.659 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: mon.c calling monitor election 2023-06-23T01:12:28.659 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:28.659 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T01:12:28.659 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T01:12:28.660 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T01:12:28.660 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-06-23T01:12:28.660 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: monmap e2: 2 mons at {a=[v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0],c=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} 2023-06-23T01:12:28.660 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: fsmap 2023-06-23T01:12:28.660 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: osdmap e4: 0 total, 0 up, 0 in 2023-06-23T01:12:28.660 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: mgrmap e13: x(active, since 46s) 2023-06-23T01:12:28.661 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: overall HEALTH_OK 2023-06-23T01:12:28.661 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:28.661 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:28.661 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:28.661 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-23T01:12:28.661 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:28.661 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:28.661 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: Deploying daemon mon.b on smithi134 2023-06-23T01:12:28.662 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T01:12:28.662 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-06-23T01:12:28.662 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 ceph-mon[116146]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-06-23T01:12:28.662 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:28 smithi134 systemd[1]: Started Ceph mon.b for 9efbfa8e-1162-11ee-9b2b-001a4aab830c. 2023-06-23T01:12:33.876 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:33 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-23T01:12:33.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:33 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T01:12:33.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:33 smithi157 ceph-mon[116945]: mon.c calling monitor election 2023-06-23T01:12:33.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:33 smithi157 ceph-mon[116945]: mon.a calling monitor election 2023-06-23T01:12:33.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:33 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T01:12:33.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:33 smithi157 ceph-mon[116945]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:33.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:33 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T01:12:33.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:33 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T01:12:33.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:33 smithi157 ceph-mon[116945]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:33.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:33 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T01:12:33.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:33 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T01:12:33.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:33 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T01:12:33.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:33 smithi157 ceph-mon[116945]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-06-23T01:12:33.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:33 smithi157 ceph-mon[116945]: monmap e3: 3 mons at {a=[v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0],b=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0],c=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} 2023-06-23T01:12:33.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:33 smithi157 ceph-mon[116945]: fsmap 2023-06-23T01:12:33.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:33 smithi157 ceph-mon[116945]: osdmap e4: 0 total, 0 up, 0 in 2023-06-23T01:12:33.879 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:33 smithi157 ceph-mon[116945]: mgrmap e13: x(active, since 54s) 2023-06-23T01:12:33.879 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:33 smithi157 ceph-mon[116945]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-06-23T01:12:33.879 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:33 smithi157 ceph-mon[116945]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-06-23T01:12:33.879 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:33 smithi157 ceph-mon[116945]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-06-23T01:12:33.879 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:33 smithi157 ceph-mon[116945]: mon.b (rank 2) addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] is down (out of quorum) 2023-06-23T01:12:33.879 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:33 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:33.879 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:33 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:34.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:33 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-23T01:12:34.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:33 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T01:12:34.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:33 smithi120 ceph-mon[79813]: mon.c calling monitor election 2023-06-23T01:12:34.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:33 smithi120 ceph-mon[79813]: mon.a calling monitor election 2023-06-23T01:12:34.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:33 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T01:12:34.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:33 smithi120 ceph-mon[79813]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:34.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:33 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T01:12:34.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:33 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T01:12:34.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:33 smithi120 ceph-mon[79813]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:34.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:33 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T01:12:34.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:33 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T01:12:34.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:33 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T01:12:34.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:33 smithi120 ceph-mon[79813]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-06-23T01:12:34.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:33 smithi120 ceph-mon[79813]: monmap e3: 3 mons at {a=[v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0],b=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0],c=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} 2023-06-23T01:12:34.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:33 smithi120 ceph-mon[79813]: fsmap 2023-06-23T01:12:34.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:33 smithi120 ceph-mon[79813]: osdmap e4: 0 total, 0 up, 0 in 2023-06-23T01:12:34.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:33 smithi120 ceph-mon[79813]: mgrmap e13: x(active, since 54s) 2023-06-23T01:12:34.013 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:33 smithi120 ceph-mon[79813]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-06-23T01:12:34.013 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:33 smithi120 ceph-mon[79813]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-06-23T01:12:34.013 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:33 smithi120 ceph-mon[79813]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-06-23T01:12:34.013 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:33 smithi120 ceph-mon[79813]: mon.b (rank 2) addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] is down (out of quorum) 2023-06-23T01:12:34.013 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:33 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:34.013 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:33 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:34.876 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:34 smithi157 ceph-mon[116945]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:34.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:34 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:34.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:34 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:34.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:34 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:12:34.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:34 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:34.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:34 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T01:12:35.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:34 smithi120 ceph-mon[79813]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:35.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:34 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:35.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:34 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:35.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:34 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:12:35.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:34 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:35.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:34 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T01:12:35.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.027+0000 7f29b546d700 1 -- 172.21.15.157:0/3925143773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f299c0061f0 msgr2=0x7f299c008650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:12:35.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.027+0000 7f29b546d700 1 --2- 172.21.15.157:0/3925143773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f299c0061f0 0x7f299c008650 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f29b00fd840 tx=0x7f29a0002040 comp rx=0 tx=0).stop 2023-06-23T01:12:35.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.027+0000 7f29b546d700 1 -- 172.21.15.157:0/3925143773 shutdown_connections 2023-06-23T01:12:35.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.027+0000 7f29b546d700 1 --2- 172.21.15.157:0/3925143773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f299c0061f0 0x7f299c008650 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:35.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.027+0000 7f29b546d700 1 -- 172.21.15.157:0/3925143773 >> 172.21.15.157:0/3925143773 conn(0x7f29b00f8340 msgr2=0x7f29b00fa750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:12:35.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.028+0000 7f29b546d700 1 -- 172.21.15.157:0/3925143773 shutdown_connections 2023-06-23T01:12:35.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.028+0000 7f29b546d700 1 -- 172.21.15.157:0/3925143773 wait complete. 2023-06-23T01:12:35.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.028+0000 7f29b546d700 1 Processor -- start 2023-06-23T01:12:35.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.028+0000 7f29b546d700 1 -- start start 2023-06-23T01:12:35.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.028+0000 7f29b546d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f299c0061f0 0x7f29b0112c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:12:35.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.028+0000 7f29b546d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29b0113150 0x7f29b010cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:12:35.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.028+0000 7f29b546d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29b010d010 0x7f29b010d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:12:35.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.028+0000 7f29b546d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b01137a0 con 0x7f29b010d010 2023-06-23T01:12:35.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.028+0000 7f29b546d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b0119a10 con 0x7f29b0113150 2023-06-23T01:12:35.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.028+0000 7f29b546d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b0119b50 con 0x7f299c0061f0 2023-06-23T01:12:35.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.029+0000 7f29aeffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f299c0061f0 0x7f29b0112c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:12:35.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.029+0000 7f29aeffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f299c0061f0 0x7f29b0112c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38938/0 (socket says 172.21.15.157:38938) 2023-06-23T01:12:35.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.029+0000 7f29aeffd700 1 -- 172.21.15.157:0/4279242138 learned_addr learned my addr 172.21.15.157:0/4279242138 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-06-23T01:12:35.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.029+0000 7f29aeffd700 1 -- 172.21.15.157:0/4279242138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f299c0061f0 msgr2=0x7f29b0112c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-06-23T01:12:35.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.029+0000 7f29aeffd700 1 -- 172.21.15.157:0/4279242138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f299c0061f0 msgr2=0x7f29b0112c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-06-23T01:12:35.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.029+0000 7f29aeffd700 1 --2- 172.21.15.157:0/4279242138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f299c0061f0 0x7f29b0112c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-06-23T01:12:35.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.029+0000 7f29aeffd700 1 --2- 172.21.15.157:0/4279242138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f299c0061f0 0x7f29b0112c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-23T01:12:35.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.029+0000 7f29ae7fc700 1 --2- 172.21.15.157:0/4279242138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29b0113150 0x7f29b010cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:12:35.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.029+0000 7f29af7fe700 1 --2- 172.21.15.157:0/4279242138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29b010d010 0x7f29b010d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:12:35.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.029+0000 7f29ae7fc700 1 -- 172.21.15.157:0/4279242138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29b0113150 msgr2=0x7f29b010cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-06-23T01:12:35.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.029+0000 7f29ae7fc700 1 -- 172.21.15.157:0/4279242138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29b0113150 msgr2=0x7f29b010cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-06-23T01:12:35.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.029+0000 7f29ae7fc700 1 --2- 172.21.15.157:0/4279242138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29b0113150 0x7f29b010cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-06-23T01:12:35.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.029+0000 7f29ae7fc700 1 --2- 172.21.15.157:0/4279242138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29b0113150 0x7f29b010cad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-23T01:12:35.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.029+0000 7f29af7fe700 1 -- 172.21.15.157:0/4279242138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f299c0061f0 msgr2=0x7f29b0112c10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-23T01:12:35.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.029+0000 7f29af7fe700 1 --2- 172.21.15.157:0/4279242138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f299c0061f0 0x7f29b0112c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:35.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.029+0000 7f29af7fe700 1 -- 172.21.15.157:0/4279242138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29b0113150 msgr2=0x7f29b010cad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-23T01:12:35.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.029+0000 7f29af7fe700 1 --2- 172.21.15.157:0/4279242138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29b0113150 0x7f29b010cad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:35.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.029+0000 7f29af7fe700 1 -- 172.21.15.157:0/4279242138 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a0005040 con 0x7f29b010d010 2023-06-23T01:12:35.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.029+0000 7f29af7fe700 1 --2- 172.21.15.157:0/4279242138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29b010d010 0x7f29b010d4b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f29a800bb00 tx=0x7f29a8009760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:12:35.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.030+0000 7f29a77fe700 1 -- 172.21.15.157:0/4279242138 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a800c8c0 con 0x7f29b010d010 2023-06-23T01:12:35.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.030+0000 7f29b546d700 1 -- 172.21.15.157:0/4279242138 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29b0119e30 con 0x7f29b010d010 2023-06-23T01:12:35.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.030+0000 7f29a77fe700 1 -- 172.21.15.157:0/4279242138 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29a80098a0 con 0x7f29b010d010 2023-06-23T01:12:35.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.030+0000 7f29a77fe700 1 -- 172.21.15.157:0/4279242138 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a80133c0 con 0x7f29b010d010 2023-06-23T01:12:35.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.030+0000 7f29b546d700 1 -- 172.21.15.157:0/4279242138 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29b011a310 con 0x7f29b010d010 2023-06-23T01:12:35.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.031+0000 7f29b546d700 1 -- 172.21.15.157:0/4279242138 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29b005f060 con 0x7f29b010d010 2023-06-23T01:12:35.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.031+0000 7f29a77fe700 1 -- 172.21.15.157:0/4279242138 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 13) v1 ==== 43276+0+0 (secure 0 0 0) 0x7f29a80145a0 con 0x7f29b010d010 2023-06-23T01:12:35.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.032+0000 7f29a77fe700 1 --2- 172.21.15.157:0/4279242138 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f2990036b30 0x7f2990038fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:12:35.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.032+0000 7f29a77fe700 1 -- 172.21.15.157:0/4279242138 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f29a8048a00 con 0x7f29b010d010 2023-06-23T01:12:35.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.032+0000 7f29aeffd700 1 --2- 172.21.15.157:0/4279242138 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f2990036b30 0x7f2990038fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:12:35.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.033+0000 7f29aeffd700 1 --2- 172.21.15.157:0/4279242138 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f2990036b30 0x7f2990038fe0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f29a0009ea0 tx=0x7f29a0024040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:12:35.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.037+0000 7f29a77fe700 1 -- 172.21.15.157:0/4279242138 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f29a801bb00 con 0x7f29b010d010 2023-06-23T01:12:35.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.311+0000 7f29b546d700 1 -- 172.21.15.157:0/4279242138 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f29b004b970 con 0x7f29b010d010 2023-06-23T01:12:35.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.311+0000 7f29a77fe700 1 -- 172.21.15.157:0/4279242138 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1273 (secure 0 0 0) 0x7f29a8047090 con 0x7f29b010d010 2023-06-23T01:12:35.318 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-06-23T01:12:35.319 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":3,"fsid":"9efbfa8e-1162-11ee-9b2b-001a4aab830c","modified":"2023-06-23T01:12:28.535950Z","created":"2023-06-23T01:10:35.325531Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:3300","nonce":0},{"type":"v1","addr":"172.21.15.120:6789","nonce":0}]},"addr":"172.21.15.120:6789/0","public_addr":"172.21.15.120:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:3300","nonce":0},{"type":"v1","addr":"172.21.15.157:6789","nonce":0}]},"addr":"172.21.15.157:6789/0","public_addr":"172.21.15.157:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-06-23T01:12:35.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.313+0000 7f29b546d700 1 -- 172.21.15.157:0/4279242138 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f2990036b30 msgr2=0x7f2990038fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:12:35.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.313+0000 7f29b546d700 1 --2- 172.21.15.157:0/4279242138 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f2990036b30 0x7f2990038fe0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f29a0009ea0 tx=0x7f29a0024040 comp rx=0 tx=0).stop 2023-06-23T01:12:35.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.313+0000 7f29b546d700 1 -- 172.21.15.157:0/4279242138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29b010d010 msgr2=0x7f29b010d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:12:35.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.313+0000 7f29b546d700 1 --2- 172.21.15.157:0/4279242138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29b010d010 0x7f29b010d4b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f29a800bb00 tx=0x7f29a8009760 comp rx=0 tx=0).stop 2023-06-23T01:12:35.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.313+0000 7f29b546d700 1 -- 172.21.15.157:0/4279242138 shutdown_connections 2023-06-23T01:12:35.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.313+0000 7f29b546d700 1 --2- 172.21.15.157:0/4279242138 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f2990036b30 0x7f2990038fe0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:35.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.313+0000 7f29b546d700 1 --2- 172.21.15.157:0/4279242138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29b010d010 0x7f29b010d4b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:35.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.314+0000 7f29b546d700 1 --2- 172.21.15.157:0/4279242138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f299c0061f0 0x7f29b0112c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:35.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.314+0000 7f29b546d700 1 --2- 172.21.15.157:0/4279242138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29b0113150 0x7f29b010cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:35.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.314+0000 7f29b546d700 1 -- 172.21.15.157:0/4279242138 >> 172.21.15.157:0/4279242138 conn(0x7f29b00f8340 msgr2=0x7f29b00fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:12:35.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.314+0000 7f29b546d700 1 -- 172.21.15.157:0/4279242138 shutdown_connections 2023-06-23T01:12:35.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:35.314+0000 7f29b546d700 1 -- 172.21.15.157:0/4279242138 wait complete. 2023-06-23T01:12:35.321 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 3 2023-06-23T01:12:35.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:35 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-23T01:12:35.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:35 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T01:12:35.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:35 smithi134 ceph-mon[116146]: mon.c calling monitor election 2023-06-23T01:12:35.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:35 smithi134 ceph-mon[116146]: mon.a calling monitor election 2023-06-23T01:12:35.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:35 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T01:12:35.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:35 smithi134 ceph-mon[116146]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:35.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:35 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T01:12:35.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:35 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T01:12:35.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:35 smithi134 ceph-mon[116146]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:35.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:35 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T01:12:35.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:35 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T01:12:35.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:35 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T01:12:35.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:35 smithi134 ceph-mon[116146]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-06-23T01:12:35.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:35 smithi134 ceph-mon[116146]: monmap e3: 3 mons at {a=[v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0],b=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0],c=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} 2023-06-23T01:12:35.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:35 smithi134 ceph-mon[116146]: fsmap 2023-06-23T01:12:35.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:35 smithi134 ceph-mon[116146]: osdmap e4: 0 total, 0 up, 0 in 2023-06-23T01:12:35.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:35 smithi134 ceph-mon[116146]: mgrmap e13: x(active, since 54s) 2023-06-23T01:12:35.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:35 smithi134 ceph-mon[116146]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-06-23T01:12:35.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:35 smithi134 ceph-mon[116146]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-06-23T01:12:35.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:35 smithi134 ceph-mon[116146]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-06-23T01:12:35.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:35 smithi134 ceph-mon[116146]: mon.b (rank 2) addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] is down (out of quorum) 2023-06-23T01:12:35.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:35 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:35.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:35 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:35.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:35 smithi134 ceph-mon[116146]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:35.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:35 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:35.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:35 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:35.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:35 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:12:35.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:35 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:35.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:35 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T01:12:36.034 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-06-23T01:12:36.035 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph config generate-minimal-conf 2023-06-23T01:12:36.830 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T01:12:36.876 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:36 smithi157 ceph-mon[116945]: mon.b calling monitor election 2023-06-23T01:12:36.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:36 smithi157 ceph-mon[116945]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:36.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:36 smithi157 ceph-mon[116945]: Updating smithi120:/etc/ceph/ceph.conf 2023-06-23T01:12:36.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:36 smithi157 ceph-mon[116945]: Updating smithi134:/etc/ceph/ceph.conf 2023-06-23T01:12:36.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:36 smithi157 ceph-mon[116945]: Updating smithi157:/etc/ceph/ceph.conf 2023-06-23T01:12:36.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:36 smithi157 ceph-mon[116945]: Updating smithi134:/var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-23T01:12:36.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:36 smithi157 ceph-mon[116945]: Updating smithi157:/var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-23T01:12:36.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:36 smithi157 ceph-mon[116945]: mon.b calling monitor election 2023-06-23T01:12:36.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:36 smithi157 ceph-mon[116945]: mon.c calling monitor election 2023-06-23T01:12:36.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:36 smithi157 ceph-mon[116945]: mon.a calling monitor election 2023-06-23T01:12:36.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:36 smithi157 ceph-mon[116945]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-06-23T01:12:36.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:36 smithi157 ceph-mon[116945]: monmap e3: 3 mons at {a=[v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0],b=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0],c=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} 2023-06-23T01:12:36.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:36 smithi157 ceph-mon[116945]: fsmap 2023-06-23T01:12:36.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:36 smithi157 ceph-mon[116945]: osdmap e4: 0 total, 0 up, 0 in 2023-06-23T01:12:36.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:36 smithi157 ceph-mon[116945]: mgrmap e13: x(active, since 56s) 2023-06-23T01:12:36.879 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:36 smithi157 ceph-mon[116945]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-06-23T01:12:36.879 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:36 smithi157 ceph-mon[116945]: Cluster is now healthy 2023-06-23T01:12:36.879 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:36 smithi157 ceph-mon[116945]: overall HEALTH_OK 2023-06-23T01:12:36.879 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:36 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:36.879 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:36 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:36.879 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:36 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:36.879 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:36 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:36.880 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:36 smithi157 ceph-mon[116945]: Updating smithi120:/var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-23T01:12:36.880 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:36 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:36.880 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:36 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:36.880 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:36 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:36.880 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:36 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-23T01:12:36.880 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:36 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-06-23T01:12:36.880 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:36 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:36.880 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:36 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T01:12:36.925 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:36 smithi120 ceph-mon[79813]: mon.b calling monitor election 2023-06-23T01:12:36.925 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:36 smithi120 ceph-mon[79813]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:36.925 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:36 smithi120 ceph-mon[79813]: Updating smithi120:/etc/ceph/ceph.conf 2023-06-23T01:12:36.925 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:36 smithi120 ceph-mon[79813]: Updating smithi134:/etc/ceph/ceph.conf 2023-06-23T01:12:36.925 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:36 smithi120 ceph-mon[79813]: Updating smithi157:/etc/ceph/ceph.conf 2023-06-23T01:12:36.926 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:36 smithi120 ceph-mon[79813]: Updating smithi134:/var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-23T01:12:36.926 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:36 smithi120 ceph-mon[79813]: Updating smithi157:/var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-23T01:12:36.926 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:36 smithi120 ceph-mon[79813]: mon.b calling monitor election 2023-06-23T01:12:36.926 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:36 smithi120 ceph-mon[79813]: mon.c calling monitor election 2023-06-23T01:12:36.926 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:36 smithi120 ceph-mon[79813]: mon.a calling monitor election 2023-06-23T01:12:36.926 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:36 smithi120 ceph-mon[79813]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-06-23T01:12:36.926 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:36 smithi120 ceph-mon[79813]: monmap e3: 3 mons at {a=[v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0],b=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0],c=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} 2023-06-23T01:12:36.927 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:36 smithi120 ceph-mon[79813]: fsmap 2023-06-23T01:12:36.927 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:36 smithi120 ceph-mon[79813]: osdmap e4: 0 total, 0 up, 0 in 2023-06-23T01:12:36.927 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:36 smithi120 ceph-mon[79813]: mgrmap e13: x(active, since 56s) 2023-06-23T01:12:36.927 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:36 smithi120 ceph-mon[79813]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-06-23T01:12:36.927 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:36 smithi120 ceph-mon[79813]: Cluster is now healthy 2023-06-23T01:12:36.927 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:36 smithi120 ceph-mon[79813]: overall HEALTH_OK 2023-06-23T01:12:36.927 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:36 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:36.927 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:36 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:36.928 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:36 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:36.928 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:36 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:36.928 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:36 smithi120 ceph-mon[79813]: Updating smithi120:/var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-23T01:12:36.928 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:36 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:36.928 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:36 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:36.928 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:36 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:36.928 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:36 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-23T01:12:36.929 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:36 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-06-23T01:12:36.929 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:36 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:36.929 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:36 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T01:12:36.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:36 smithi134 ceph-mon[116146]: mon.b calling monitor election 2023-06-23T01:12:36.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:36 smithi134 ceph-mon[116146]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:36.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:36 smithi134 ceph-mon[116146]: Updating smithi120:/etc/ceph/ceph.conf 2023-06-23T01:12:36.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:36 smithi134 ceph-mon[116146]: Updating smithi134:/etc/ceph/ceph.conf 2023-06-23T01:12:36.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:36 smithi134 ceph-mon[116146]: Updating smithi157:/etc/ceph/ceph.conf 2023-06-23T01:12:36.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:36 smithi134 ceph-mon[116146]: Updating smithi134:/var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-23T01:12:36.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:36 smithi134 ceph-mon[116146]: Updating smithi157:/var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-23T01:12:36.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:36 smithi134 ceph-mon[116146]: mon.b calling monitor election 2023-06-23T01:12:36.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:36 smithi134 ceph-mon[116146]: mon.c calling monitor election 2023-06-23T01:12:36.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:36 smithi134 ceph-mon[116146]: mon.a calling monitor election 2023-06-23T01:12:36.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:36 smithi134 ceph-mon[116146]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-06-23T01:12:36.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:36 smithi134 ceph-mon[116146]: monmap e3: 3 mons at {a=[v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0],b=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0],c=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} 2023-06-23T01:12:36.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:36 smithi134 ceph-mon[116146]: fsmap 2023-06-23T01:12:36.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:36 smithi134 ceph-mon[116146]: osdmap e4: 0 total, 0 up, 0 in 2023-06-23T01:12:36.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:36 smithi134 ceph-mon[116146]: mgrmap e13: x(active, since 56s) 2023-06-23T01:12:36.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:36 smithi134 ceph-mon[116146]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-06-23T01:12:36.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:36 smithi134 ceph-mon[116146]: Cluster is now healthy 2023-06-23T01:12:36.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:36 smithi134 ceph-mon[116146]: overall HEALTH_OK 2023-06-23T01:12:36.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:36 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:36.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:36 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:36.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:36 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:36.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:36 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:36.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:36 smithi134 ceph-mon[116146]: Updating smithi120:/var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-23T01:12:36.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:36 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:36.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:36 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:36.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:36 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:36.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:36 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-23T01:12:36.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:36 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-06-23T01:12:36.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:36 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:36.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:36 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T01:12:38.183 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:37 smithi134 ceph-mon[116146]: Reconfiguring mon.a (unknown last config time)... 2023-06-23T01:12:38.184 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:37 smithi134 ceph-mon[116146]: Reconfiguring daemon mon.a on smithi120 2023-06-23T01:12:38.184 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:37 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:38.184 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:37 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:38.184 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:37 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-23T01:12:38.184 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:37 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-06-23T01:12:38.185 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:37 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:38.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:37 smithi120 ceph-mon[79813]: Reconfiguring mon.a (unknown last config time)... 2023-06-23T01:12:38.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:37 smithi120 ceph-mon[79813]: Reconfiguring daemon mon.a on smithi120 2023-06-23T01:12:38.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:37 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:38.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:37 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:38.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:37 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-23T01:12:38.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:37 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-06-23T01:12:38.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:37 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:38.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:37 smithi157 ceph-mon[116945]: Reconfiguring mon.a (unknown last config time)... 2023-06-23T01:12:38.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:37 smithi157 ceph-mon[116945]: Reconfiguring daemon mon.a on smithi120 2023-06-23T01:12:38.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:37 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:38.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:37 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:38.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:37 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-23T01:12:38.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:37 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-06-23T01:12:38.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:37 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:38.818 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.815+0000 7f0263e6b700 1 -- 172.21.15.120:0/4123429041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f025c0ff520 msgr2=0x7f025c10bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:12:38.819 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.815+0000 7f0263e6b700 1 --2- 172.21.15.120:0/4123429041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f025c0ff520 0x7f025c10bc10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0254009d90 tx=0x7f0254004d10 comp rx=0 tx=0).stop 2023-06-23T01:12:38.819 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.815+0000 7f0263e6b700 1 -- 172.21.15.120:0/4123429041 shutdown_connections 2023-06-23T01:12:38.819 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.815+0000 7f0263e6b700 1 --2- 172.21.15.120:0/4123429041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f025c0ff520 0x7f025c10bc10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:38.819 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.815+0000 7f0263e6b700 1 --2- 172.21.15.120:0/4123429041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f025c104e30 0x7f025c105200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:38.819 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.815+0000 7f0263e6b700 1 --2- 172.21.15.120:0/4123429041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f025c1057d0 0x7f025c0fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:38.819 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.815+0000 7f0263e6b700 1 -- 172.21.15.120:0/4123429041 >> 172.21.15.120:0/4123429041 conn(0x7f025c0fa9a0 msgr2=0x7f025c0fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:12:38.820 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.815+0000 7f0263e6b700 1 -- 172.21.15.120:0/4123429041 shutdown_connections 2023-06-23T01:12:38.820 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.815+0000 7f0263e6b700 1 -- 172.21.15.120:0/4123429041 wait complete. 2023-06-23T01:12:38.821 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.816+0000 7f0263e6b700 1 Processor -- start 2023-06-23T01:12:38.821 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.816+0000 7f0263e6b700 1 -- start start 2023-06-23T01:12:38.821 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.816+0000 7f0263e6b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f025c0ff520 0x7f025c115900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:12:38.822 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.816+0000 7f0263e6b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f025c104e30 0x7f025c115e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:12:38.822 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.816+0000 7f0263e6b700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f025c1057d0 0x7f025c10fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:12:38.822 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.816+0000 7f0263e6b700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c110090 con 0x7f025c1057d0 2023-06-23T01:12:38.822 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.816+0000 7f0263e6b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c110200 con 0x7f025c0ff520 2023-06-23T01:12:38.822 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.816+0000 7f0263e6b700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c1103c0 con 0x7f025c104e30 2023-06-23T01:12:38.822 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.817+0000 7f0262408700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f025c1057d0 0x7f025c10fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:12:38.822 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.817+0000 7f0261406700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f025c104e30 0x7f025c115e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:12:38.823 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.817+0000 7f0262408700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f025c1057d0 0x7f025c10fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40712/0 (socket says 172.21.15.120:40712) 2023-06-23T01:12:38.823 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.817+0000 7f0262408700 1 -- 172.21.15.120:0/4259483824 learned_addr learned my addr 172.21.15.120:0/4259483824 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:12:38.823 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.817+0000 7f0261c07700 1 --2- 172.21.15.120:0/4259483824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f025c0ff520 0x7f025c115900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:12:38.823 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.817+0000 7f0262408700 1 -- 172.21.15.120:0/4259483824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f025c104e30 msgr2=0x7f025c115e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:12:38.823 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.817+0000 7f0262408700 1 --2- 172.21.15.120:0/4259483824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f025c104e30 0x7f025c115e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:38.823 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.817+0000 7f0262408700 1 -- 172.21.15.120:0/4259483824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f025c0ff520 msgr2=0x7f025c115900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:12:38.823 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.817+0000 7f0262408700 1 --2- 172.21.15.120:0/4259483824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f025c0ff520 0x7f025c115900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:38.823 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.817+0000 7f0262408700 1 -- 172.21.15.120:0/4259483824 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f024c005040 con 0x7f025c1057d0 2023-06-23T01:12:38.824 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.817+0000 7f0262408700 1 --2- 172.21.15.120:0/4259483824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f025c1057d0 0x7f025c10fa20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f025400b700 tx=0x7f0254004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:12:38.824 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.818+0000 7f02527fc700 1 -- 172.21.15.120:0/4259483824 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025400e030 con 0x7f025c1057d0 2023-06-23T01:12:38.824 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.818+0000 7f02527fc700 1 -- 172.21.15.120:0/4259483824 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0254008ec0 con 0x7f025c1057d0 2023-06-23T01:12:38.824 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.818+0000 7f0263e6b700 1 -- 172.21.15.120:0/4259483824 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0254009940 con 0x7f025c1057d0 2023-06-23T01:12:38.824 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.818+0000 7f02527fc700 1 -- 172.21.15.120:0/4259483824 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0254012680 con 0x7f025c1057d0 2023-06-23T01:12:38.824 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.818+0000 7f0263e6b700 1 -- 172.21.15.120:0/4259483824 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f025c11c700 con 0x7f025c1057d0 2023-06-23T01:12:38.825 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.819+0000 7f02527fc700 1 -- 172.21.15.120:0/4259483824 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 13) v1 ==== 43276+0+0 (secure 0 0 0) 0x7f0254012ab0 con 0x7f025c1057d0 2023-06-23T01:12:38.825 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.819+0000 7f0263e6b700 1 -- 172.21.15.120:0/4259483824 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f025c04c1b0 con 0x7f025c1057d0 2023-06-23T01:12:38.825 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.819+0000 7f02527fc700 1 --2- 172.21.15.120:0/4259483824 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f0248036890 0x7f0248038d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:12:38.826 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.819+0000 7f02527fc700 1 -- 172.21.15.120:0/4259483824 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f025404a450 con 0x7f025c1057d0 2023-06-23T01:12:38.826 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.820+0000 7f0261c07700 1 --2- 172.21.15.120:0/4259483824 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f0248036890 0x7f0248038d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:12:38.826 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.820+0000 7f0261c07700 1 --2- 172.21.15.120:0/4259483824 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f0248036890 0x7f0248038d40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0258004f60 tx=0x7f025800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:12:38.827 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:38.824+0000 7f02527fc700 1 -- 172.21.15.120:0/4259483824 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f025401c440 con 0x7f025c1057d0 2023-06-23T01:12:39.020 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:39.017+0000 7f0263e6b700 1 -- 172.21.15.120:0/4259483824 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f025c10ecc0 con 0x7f025c1057d0 2023-06-23T01:12:39.021 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:39.018+0000 7f02527fc700 1 -- 172.21.15.120:0/4259483824 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+277 (secure 0 0 0) 0x7f0254017070 con 0x7f025c1057d0 2023-06-23T01:12:39.021 INFO:teuthology.orchestra.run.smithi120.stdout:# minimal ceph.conf for 9efbfa8e-1162-11ee-9b2b-001a4aab830c 2023-06-23T01:12:39.021 INFO:teuthology.orchestra.run.smithi120.stdout:[global] 2023-06-23T01:12:39.021 INFO:teuthology.orchestra.run.smithi120.stdout: fsid = 9efbfa8e-1162-11ee-9b2b-001a4aab830c 2023-06-23T01:12:39.022 INFO:teuthology.orchestra.run.smithi120.stdout: mon_host = [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] 2023-06-23T01:12:39.023 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:39.020+0000 7f0263e6b700 1 -- 172.21.15.120:0/4259483824 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f0248036890 msgr2=0x7f0248038d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:12:39.023 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:39.020+0000 7f0263e6b700 1 --2- 172.21.15.120:0/4259483824 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f0248036890 0x7f0248038d40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0258004f60 tx=0x7f025800b040 comp rx=0 tx=0).stop 2023-06-23T01:12:39.023 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:39.020+0000 7f0263e6b700 1 -- 172.21.15.120:0/4259483824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f025c1057d0 msgr2=0x7f025c10fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:12:39.023 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:39.020+0000 7f0263e6b700 1 --2- 172.21.15.120:0/4259483824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f025c1057d0 0x7f025c10fa20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f025400b700 tx=0x7f0254004d10 comp rx=0 tx=0).stop 2023-06-23T01:12:39.024 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:39.020+0000 7f0263e6b700 1 -- 172.21.15.120:0/4259483824 shutdown_connections 2023-06-23T01:12:39.024 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:39.020+0000 7f0263e6b700 1 --2- 172.21.15.120:0/4259483824 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f0248036890 0x7f0248038d40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:39.024 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:39.020+0000 7f0263e6b700 1 --2- 172.21.15.120:0/4259483824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f025c1057d0 0x7f025c10fa20 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:39.024 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:39.020+0000 7f0263e6b700 1 --2- 172.21.15.120:0/4259483824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f025c0ff520 0x7f025c115900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:39.024 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:39.020+0000 7f0263e6b700 1 --2- 172.21.15.120:0/4259483824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f025c104e30 0x7f025c115e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:39.024 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:39.020+0000 7f0263e6b700 1 -- 172.21.15.120:0/4259483824 >> 172.21.15.120:0/4259483824 conn(0x7f025c0fa9a0 msgr2=0x7f025c1029f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:12:39.024 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:39.020+0000 7f0263e6b700 1 -- 172.21.15.120:0/4259483824 shutdown_connections 2023-06-23T01:12:39.025 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:39.020+0000 7f0263e6b700 1 -- 172.21.15.120:0/4259483824 wait complete. 2023-06-23T01:12:39.193 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:38 smithi134 ceph-mon[116146]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:39.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:38 smithi134 ceph-mon[116146]: Reconfiguring mon.b (monmap changed)... 2023-06-23T01:12:39.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:38 smithi134 ceph-mon[116146]: Reconfiguring daemon mon.b on smithi134 2023-06-23T01:12:39.219 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:38 smithi120 ceph-mon[79813]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:39.219 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:38 smithi120 ceph-mon[79813]: Reconfiguring mon.b (monmap changed)... 2023-06-23T01:12:39.220 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:38 smithi120 ceph-mon[79813]: Reconfiguring daemon mon.b on smithi134 2023-06-23T01:12:39.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:38 smithi157 ceph-mon[116945]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:39.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:38 smithi157 ceph-mon[116945]: Reconfiguring mon.b (monmap changed)... 2023-06-23T01:12:39.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:38 smithi157 ceph-mon[116945]: Reconfiguring daemon mon.b on smithi134 2023-06-23T01:12:39.494 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-06-23T01:12:39.495 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2023-06-23T01:12:39.495 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/ceph/ceph.conf 2023-06-23T01:12:39.558 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2023-06-23T01:12:39.559 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-06-23T01:12:39.659 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-06-23T01:12:39.659 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.conf 2023-06-23T01:12:39.692 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-06-23T01:12:39.693 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-06-23T01:12:39.761 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-06-23T01:12:39.761 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.conf 2023-06-23T01:12:39.788 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-06-23T01:12:39.788 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-06-23T01:12:39.852 INFO:tasks.cephadm:Adding mgr.x on smithi120 2023-06-23T01:12:39.852 INFO:tasks.cephadm:Adding mgr.y on smithi134 2023-06-23T01:12:39.852 INFO:tasks.cephadm:Adding mgr.z on smithi157 2023-06-23T01:12:39.853 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph orch apply mgr '3;smithi120=x;smithi134=y;smithi157=z' 2023-06-23T01:12:39.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:39 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/4259483824' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:39.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:39 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:39.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:39 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:39.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:39 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-23T01:12:39.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:39 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-06-23T01:12:39.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:39 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:40.059 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:39 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/4259483824' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:40.060 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:39 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:40.060 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:39 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:40.060 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:39 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-23T01:12:40.060 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:39 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-06-23T01:12:40.060 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:39 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:40.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:39 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/4259483824' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:40.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:39 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:40.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:39 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:40.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:39 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-23T01:12:40.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:39 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-06-23T01:12:40.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:39 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:40.695 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.c/config 2023-06-23T01:12:41.193 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:40 smithi157 ceph-mon[116945]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:41.193 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:40 smithi157 ceph-mon[116945]: Reconfiguring mon.c (monmap changed)... 2023-06-23T01:12:41.193 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:40 smithi157 ceph-mon[116945]: Reconfiguring daemon mon.c on smithi157 2023-06-23T01:12:41.193 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:40 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:41.193 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:40 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:41.194 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:40 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:12:41.194 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:40 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:41.194 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:40 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:12:41.194 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:40 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:41.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:40 smithi134 ceph-mon[116146]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:41.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:40 smithi134 ceph-mon[116146]: Reconfiguring mon.c (monmap changed)... 2023-06-23T01:12:41.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:40 smithi134 ceph-mon[116146]: Reconfiguring daemon mon.c on smithi157 2023-06-23T01:12:41.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:40 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:41.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:40 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:41.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:40 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:12:41.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:40 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:41.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:40 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:12:41.196 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:40 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:41.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:40 smithi120 ceph-mon[79813]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:41.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:40 smithi120 ceph-mon[79813]: Reconfiguring mon.c (monmap changed)... 2023-06-23T01:12:41.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:40 smithi120 ceph-mon[79813]: Reconfiguring daemon mon.c on smithi157 2023-06-23T01:12:41.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:40 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:41.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:40 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:41.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:40 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:12:41.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:40 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:41.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:40 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:12:41.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:40 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:42.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.047+0000 7fec634ef700 1 -- 172.21.15.157:0/1434697367 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec5c0fe820 msgr2=0x7fec5c1050e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:12:42.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.047+0000 7fec634ef700 1 --2- 172.21.15.157:0/1434697367 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec5c0fe820 0x7fec5c1050e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fec54009d60 tx=0x7fec54004d10 comp rx=0 tx=0).stop 2023-06-23T01:12:42.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.047+0000 7fec634ef700 1 -- 172.21.15.157:0/1434697367 shutdown_connections 2023-06-23T01:12:42.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.047+0000 7fec634ef700 1 --2- 172.21.15.157:0/1434697367 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec5c0fe820 0x7fec5c1050e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:42.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.047+0000 7fec634ef700 1 --2- 172.21.15.157:0/1434697367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec5c0fcc80 0x7fec5c0fd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:42.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.047+0000 7fec634ef700 1 --2- 172.21.15.157:0/1434697367 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec5c0fde90 0x7fec5c0fe2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:42.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.047+0000 7fec634ef700 1 -- 172.21.15.157:0/1434697367 >> 172.21.15.157:0/1434697367 conn(0x7fec5c0f83a0 msgr2=0x7fec5c0fa810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:12:42.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.047+0000 7fec634ef700 1 -- 172.21.15.157:0/1434697367 shutdown_connections 2023-06-23T01:12:42.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.047+0000 7fec634ef700 1 -- 172.21.15.157:0/1434697367 wait complete. 2023-06-23T01:12:42.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.048+0000 7fec634ef700 1 Processor -- start 2023-06-23T01:12:42.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.048+0000 7fec634ef700 1 -- start start 2023-06-23T01:12:42.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.048+0000 7fec634ef700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec5c0fcc80 0x7fec5c10d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:12:42.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.048+0000 7fec634ef700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec5c0fde90 0x7fec5c10dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:12:42.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.048+0000 7fec634ef700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec5c0fe820 0x7fec5c114cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:12:42.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.048+0000 7fec634ef700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5c115300 con 0x7fec5c0fcc80 2023-06-23T01:12:42.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.048+0000 7fec634ef700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5c115470 con 0x7fec5c0fde90 2023-06-23T01:12:42.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.048+0000 7fec634ef700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5c115630 con 0x7fec5c0fe820 2023-06-23T01:12:42.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.048+0000 7fec61a8c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec5c0fe820 0x7fec5c114cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:12:42.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.048+0000 7fec61a8c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec5c0fe820 0x7fec5c114cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38954/0 (socket says 172.21.15.157:38954) 2023-06-23T01:12:42.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.048+0000 7fec61a8c700 1 -- 172.21.15.157:0/4175133285 learned_addr learned my addr 172.21.15.157:0/4175133285 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-06-23T01:12:42.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.048+0000 7fec60a8a700 1 --2- 172.21.15.157:0/4175133285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec5c0fde90 0x7fec5c10dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:12:42.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.048+0000 7fec61a8c700 1 -- 172.21.15.157:0/4175133285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec5c0fde90 msgr2=0x7fec5c10dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:12:42.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.048+0000 7fec61a8c700 1 --2- 172.21.15.157:0/4175133285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec5c0fde90 0x7fec5c10dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:42.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.048+0000 7fec61a8c700 1 -- 172.21.15.157:0/4175133285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec5c0fcc80 msgr2=0x7fec5c10d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:12:42.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.048+0000 7fec61a8c700 1 --2- 172.21.15.157:0/4175133285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec5c0fcc80 0x7fec5c10d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:42.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.048+0000 7fec61a8c700 1 -- 172.21.15.157:0/4175133285 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec54009940 con 0x7fec5c0fe820 2023-06-23T01:12:42.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.049+0000 7fec61a8c700 1 --2- 172.21.15.157:0/4175133285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec5c0fe820 0x7fec5c114cf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fec54015040 tx=0x7fec54004d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:12:42.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.049+0000 7fec51ffb700 1 -- 172.21.15.157:0/4175133285 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5400e070 con 0x7fec5c0fe820 2023-06-23T01:12:42.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.049+0000 7fec51ffb700 1 -- 172.21.15.157:0/4175133285 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec54012d00 con 0x7fec5c0fe820 2023-06-23T01:12:42.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.049+0000 7fec634ef700 1 -- 172.21.15.157:0/4175133285 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec5c1158b0 con 0x7fec5c0fe820 2023-06-23T01:12:42.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.049+0000 7fec51ffb700 1 -- 172.21.15.157:0/4175133285 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5401cad0 con 0x7fec5c0fe820 2023-06-23T01:12:42.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.049+0000 7fec634ef700 1 -- 172.21.15.157:0/4175133285 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec5c115d60 con 0x7fec5c0fe820 2023-06-23T01:12:42.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.050+0000 7fec51ffb700 1 -- 172.21.15.157:0/4175133285 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 13) v1 ==== 43276+0+0 (secure 0 0 0) 0x7fec54012830 con 0x7fec5c0fe820 2023-06-23T01:12:42.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.050+0000 7fec51ffb700 1 --2- 172.21.15.157:0/4175133285 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fec48036890 0x7fec48038d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:12:42.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.050+0000 7fec51ffb700 1 -- 172.21.15.157:0/4175133285 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fec5404a190 con 0x7fec5c0fe820 2023-06-23T01:12:42.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.050+0000 7fec634ef700 1 -- 172.21.15.157:0/4175133285 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec5c1038c0 con 0x7fec5c0fe820 2023-06-23T01:12:42.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.050+0000 7fec6128b700 1 --2- 172.21.15.157:0/4175133285 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fec48036890 0x7fec48038d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:12:42.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.051+0000 7fec6128b700 1 --2- 172.21.15.157:0/4175133285 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fec48036890 0x7fec48038d40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fec5c0fdcf0 tx=0x7fec58007ce0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:12:42.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.055+0000 7fec51ffb700 1 -- 172.21.15.157:0/4175133285 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fec54017020 con 0x7fec5c0fe820 2023-06-23T01:12:42.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.248+0000 7fec634ef700 1 -- 172.21.15.157:0/4175133285 --> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi120=x;smithi134=y;smithi157=z", "target": ["mon-mgr", ""]}) v1 -- 0x7fec5c02b0d0 con 0x7fec48036890 2023-06-23T01:12:42.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.254+0000 7fec51ffb700 1 -- 172.21.15.157:0/4175133285 <== mgr.14152 v2:172.21.15.120:6800/1770912857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fec5c02b0d0 con 0x7fec48036890 2023-06-23T01:12:42.258 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled mgr update... 2023-06-23T01:12:42.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.257+0000 7fec634ef700 1 -- 172.21.15.157:0/4175133285 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fec48036890 msgr2=0x7fec48038d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:12:42.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.257+0000 7fec634ef700 1 --2- 172.21.15.157:0/4175133285 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fec48036890 0x7fec48038d40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fec5c0fdcf0 tx=0x7fec58007ce0 comp rx=0 tx=0).stop 2023-06-23T01:12:42.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.257+0000 7fec634ef700 1 -- 172.21.15.157:0/4175133285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec5c0fe820 msgr2=0x7fec5c114cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:12:42.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.257+0000 7fec634ef700 1 --2- 172.21.15.157:0/4175133285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec5c0fe820 0x7fec5c114cf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fec54015040 tx=0x7fec54004d10 comp rx=0 tx=0).stop 2023-06-23T01:12:42.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.257+0000 7fec634ef700 1 -- 172.21.15.157:0/4175133285 shutdown_connections 2023-06-23T01:12:42.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.257+0000 7fec634ef700 1 --2- 172.21.15.157:0/4175133285 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fec48036890 0x7fec48038d40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:42.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.257+0000 7fec634ef700 1 --2- 172.21.15.157:0/4175133285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec5c0fcc80 0x7fec5c10d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:42.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.257+0000 7fec634ef700 1 --2- 172.21.15.157:0/4175133285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec5c0fe820 0x7fec5c114cf0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:42.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.257+0000 7fec634ef700 1 --2- 172.21.15.157:0/4175133285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec5c0fde90 0x7fec5c10dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:42.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.257+0000 7fec634ef700 1 -- 172.21.15.157:0/4175133285 >> 172.21.15.157:0/4175133285 conn(0x7fec5c0f83a0 msgr2=0x7fec5c0fa380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:12:42.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.257+0000 7fec634ef700 1 -- 172.21.15.157:0/4175133285 shutdown_connections 2023-06-23T01:12:42.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:12:42.257+0000 7fec634ef700 1 -- 172.21.15.157:0/4175133285 wait complete. 2023-06-23T01:12:43.031 DEBUG:teuthology.orchestra.run.smithi134:mgr.y> sudo journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mgr.y.service 2023-06-23T01:12:43.034 DEBUG:teuthology.orchestra.run.smithi157:mgr.z> sudo journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mgr.z.service 2023-06-23T01:12:43.037 INFO:tasks.cephadm:Deploying OSDs... 2023-06-23T01:12:43.037 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2023-06-23T01:12:43.037 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/scratch_devs of=/dev/stdout 2023-06-23T01:12:43.062 INFO:journalctl@ceph.mgr.z.smithi157.stdout:-- Logs begin at Fri 2023-06-23 00:49:55 UTC. -- 2023-06-23T01:12:43.063 INFO:journalctl@ceph.mgr.y.smithi134.stdout:-- Logs begin at Fri 2023-06-23 00:49:19 UTC. -- 2023-06-23T01:12:43.077 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-06-23T01:12:43.077 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_1 2023-06-23T01:12:43.156 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-06-23T01:12:43.156 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T01:12:43.157 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 452 Links: 1 2023-06-23T01:12:43.157 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T01:12:43.157 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T01:12:43.158 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2023-06-23 01:11:48.647371446 +0000 2023-06-23T01:12:43.158 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2023-06-23 01:04:31.007274881 +0000 2023-06-23T01:12:43.158 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2023-06-23 01:04:31.007274881 +0000 2023-06-23T01:12:43.158 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: - 2023-06-23T01:12:43.159 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-06-23T01:12:43.183 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:42 smithi157 ceph-mon[116945]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:43.184 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:42 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:43.184 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:42 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:12:43.184 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:42 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:43.184 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:42 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:12:43.184 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:42 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:43.185 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:42 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-06-23T01:12:43.185 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:42 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-06-23T01:12:43.185 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:42 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-06-23T01:12:43.185 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:42 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:43.193 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:42 smithi134 ceph-mon[116146]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:43.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:42 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:43.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:42 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:12:43.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:42 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:43.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:42 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:12:43.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:42 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:43.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:42 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-06-23T01:12:43.196 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:42 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-06-23T01:12:43.196 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:42 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-06-23T01:12:43.196 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:42 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:43.249 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:42 smithi120 ceph-mon[79813]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:43.249 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:42 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:43.250 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:42 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:12:43.250 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:42 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:43.250 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:42 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:12:43.251 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:42 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:43.251 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:42 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-06-23T01:12:43.251 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:42 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-06-23T01:12:43.251 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:42 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-06-23T01:12:43.251 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:42 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:43.255 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2023-06-23T01:12:43.255 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2023-06-23T01:12:43.255 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000339382 s, 1.5 MB/s 2023-06-23T01:12:43.258 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-06-23T01:12:43.296 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_2 2023-06-23T01:12:43.373 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-06-23T01:12:43.373 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T01:12:43.374 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 458 Links: 1 2023-06-23T01:12:43.374 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T01:12:43.374 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T01:12:43.374 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2023-06-23 01:11:48.647371446 +0000 2023-06-23T01:12:43.375 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2023-06-23 01:04:31.807257074 +0000 2023-06-23T01:12:43.382 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2023-06-23 01:04:31.807257074 +0000 2023-06-23T01:12:43.382 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: - 2023-06-23T01:12:43.382 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-06-23T01:12:43.470 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2023-06-23T01:12:43.470 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2023-06-23T01:12:43.470 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.00030807 s, 1.7 MB/s 2023-06-23T01:12:43.472 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-06-23T01:12:43.509 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_3 2023-06-23T01:12:43.586 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-06-23T01:12:43.586 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T01:12:43.586 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 464 Links: 1 2023-06-23T01:12:43.586 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T01:12:43.587 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T01:12:43.587 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2023-06-23 01:11:48.648371424 +0000 2023-06-23T01:12:43.587 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2023-06-23 01:04:32.599239447 +0000 2023-06-23T01:12:43.587 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2023-06-23 01:04:32.599239447 +0000 2023-06-23T01:12:43.587 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: - 2023-06-23T01:12:43.587 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-06-23T01:12:43.683 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2023-06-23T01:12:43.683 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2023-06-23T01:12:43.684 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000323489 s, 1.6 MB/s 2023-06-23T01:12:43.685 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-06-23T01:12:43.723 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_4 2023-06-23T01:12:43.799 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-06-23T01:12:43.800 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T01:12:43.800 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 470 Links: 1 2023-06-23T01:12:43.800 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T01:12:43.800 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T01:12:43.800 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2023-06-23 01:11:48.648371424 +0000 2023-06-23T01:12:43.800 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2023-06-23 01:04:33.392221796 +0000 2023-06-23T01:12:43.800 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2023-06-23 01:04:33.392221796 +0000 2023-06-23T01:12:43.801 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: - 2023-06-23T01:12:43.801 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-06-23T01:12:43.896 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2023-06-23T01:12:43.897 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2023-06-23T01:12:43.897 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000317059 s, 1.6 MB/s 2023-06-23T01:12:43.899 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-06-23T01:12:43.937 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-06-23T01:12:43.937 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/scratch_devs of=/dev/stdout 2023-06-23T01:12:43.955 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-06-23T01:12:43.956 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_1 2023-06-23T01:12:44.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:43 smithi120 ceph-mon[79813]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi120=x;smithi134=y;smithi157=z", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:12:44.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:43 smithi120 ceph-mon[79813]: Saving service mgr spec with placement smithi120=x;smithi134=y;smithi157=z;count:3 2023-06-23T01:12:44.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:43 smithi120 ceph-mon[79813]: Deploying daemon mgr.z on smithi157 2023-06-23T01:12:44.013 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-06-23T01:12:44.013 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T01:12:44.013 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 6h/6d Inode: 216335 Links: 1 2023-06-23T01:12:44.013 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T01:12:44.013 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T01:12:44.014 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2023-06-23 01:12:27.945314479 +0000 2023-06-23T01:12:44.014 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2023-06-23 01:02:10.230274313 +0000 2023-06-23T01:12:44.014 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2023-06-23 01:02:10.230274313 +0000 2023-06-23T01:12:44.014 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: - 2023-06-23T01:12:44.014 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-06-23T01:12:44.080 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2023-06-23T01:12:44.080 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2023-06-23T01:12:44.080 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000267571 s, 1.9 MB/s 2023-06-23T01:12:44.081 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-06-23T01:12:44.140 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_2 2023-06-23T01:12:44.193 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:43 smithi134 ceph-mon[116146]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi120=x;smithi134=y;smithi157=z", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:12:44.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:43 smithi134 ceph-mon[116146]: Saving service mgr spec with placement smithi120=x;smithi134=y;smithi157=z;count:3 2023-06-23T01:12:44.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:43 smithi134 ceph-mon[116146]: Deploying daemon mgr.z on smithi157 2023-06-23T01:12:44.197 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-06-23T01:12:44.198 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T01:12:44.198 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 6h/6d Inode: 215550 Links: 1 2023-06-23T01:12:44.198 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T01:12:44.198 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T01:12:44.198 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2023-06-23 01:12:27.945314479 +0000 2023-06-23T01:12:44.199 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2023-06-23 01:02:10.740262467 +0000 2023-06-23T01:12:44.199 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2023-06-23 01:02:10.740262467 +0000 2023-06-23T01:12:44.199 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: - 2023-06-23T01:12:44.199 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-06-23T01:12:44.263 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2023-06-23T01:12:44.264 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2023-06-23T01:12:44.264 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000257929 s, 2.0 MB/s 2023-06-23T01:12:44.265 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-06-23T01:12:44.322 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_3 2023-06-23T01:12:44.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:43 smithi157 ceph-mon[116945]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi120=x;smithi134=y;smithi157=z", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:12:44.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:43 smithi157 ceph-mon[116945]: Saving service mgr spec with placement smithi120=x;smithi134=y;smithi157=z;count:3 2023-06-23T01:12:44.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:43 smithi157 ceph-mon[116945]: Deploying daemon mgr.z on smithi157 2023-06-23T01:12:44.377 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Jun 23 01:12:44 smithi157 systemd[1]: Starting Ceph mgr.z for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:12:44.378 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-06-23T01:12:44.379 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T01:12:44.379 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 6h/6d Inode: 214618 Links: 1 2023-06-23T01:12:44.379 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T01:12:44.379 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T01:12:44.379 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2023-06-23 01:12:27.945314479 +0000 2023-06-23T01:12:44.379 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2023-06-23 01:02:11.237250921 +0000 2023-06-23T01:12:44.379 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2023-06-23 01:02:11.237250921 +0000 2023-06-23T01:12:44.380 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: - 2023-06-23T01:12:44.380 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-06-23T01:12:44.444 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2023-06-23T01:12:44.444 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2023-06-23T01:12:44.444 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000259772 s, 2.0 MB/s 2023-06-23T01:12:44.445 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-06-23T01:12:44.503 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_4 2023-06-23T01:12:44.559 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-06-23T01:12:44.560 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T01:12:44.560 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 6h/6d Inode: 213471 Links: 1 2023-06-23T01:12:44.560 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T01:12:44.561 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T01:12:44.561 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2023-06-23 01:12:27.945314479 +0000 2023-06-23T01:12:44.561 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2023-06-23 01:02:11.732239422 +0000 2023-06-23T01:12:44.561 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2023-06-23 01:02:11.732239422 +0000 2023-06-23T01:12:44.561 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: - 2023-06-23T01:12:44.561 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-06-23T01:12:44.624 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2023-06-23T01:12:44.624 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2023-06-23T01:12:44.624 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000201281 s, 2.5 MB/s 2023-06-23T01:12:44.625 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-06-23T01:12:44.681 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-06-23T01:12:44.682 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/scratch_devs of=/dev/stdout 2023-06-23T01:12:44.699 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-06-23T01:12:44.699 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_1 2023-06-23T01:12:44.754 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-06-23T01:12:44.754 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T01:12:44.755 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 6h/6d Inode: 214261 Links: 1 2023-06-23T01:12:44.755 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T01:12:44.755 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T01:12:44.755 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2023-06-23 01:12:44.108039772 +0000 2023-06-23T01:12:44.755 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2023-06-23 01:06:29.098136507 +0000 2023-06-23T01:12:44.755 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2023-06-23 01:06:29.098136507 +0000 2023-06-23T01:12:44.755 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: - 2023-06-23T01:12:44.756 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-06-23T01:12:44.769 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Jun 23 01:12:44 smithi157 bash[119208]: e644d149231d71f9e560a92153fc62659c6bd1842879f81ffa2ea962b0aba6c6 2023-06-23T01:12:44.770 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Jun 23 01:12:44 smithi157 ceph-mgr[119288]: -- 172.21.15.157:0/3051776426 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f8ab76a9c0 con 0x55f8ab741000 2023-06-23T01:12:44.796 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-06-23T01:12:44.797 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-06-23T01:12:44.797 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000105199 s, 4.9 MB/s 2023-06-23T01:12:44.798 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-06-23T01:12:44.853 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_2 2023-06-23T01:12:44.907 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-06-23T01:12:44.908 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T01:12:44.908 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 6h/6d Inode: 209880 Links: 1 2023-06-23T01:12:44.908 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T01:12:44.908 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T01:12:44.908 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2023-06-23 01:12:44.108039772 +0000 2023-06-23T01:12:44.908 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2023-06-23 01:06:29.604127145 +0000 2023-06-23T01:12:44.908 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2023-06-23 01:06:29.604127145 +0000 2023-06-23T01:12:44.909 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: - 2023-06-23T01:12:44.909 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-06-23T01:12:44.970 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-06-23T01:12:44.971 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-06-23T01:12:44.971 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000108058 s, 4.7 MB/s 2023-06-23T01:12:44.972 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-06-23T01:12:45.027 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_3 2023-06-23T01:12:45.081 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-06-23T01:12:45.081 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T01:12:45.081 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 6h/6d Inode: 214386 Links: 1 2023-06-23T01:12:45.082 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T01:12:45.082 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T01:12:45.082 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2023-06-23 01:12:44.109039754 +0000 2023-06-23T01:12:45.082 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2023-06-23 01:06:30.095118061 +0000 2023-06-23T01:12:45.082 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2023-06-23 01:06:30.095118061 +0000 2023-06-23T01:12:45.082 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: - 2023-06-23T01:12:45.082 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-06-23T01:12:45.126 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Jun 23 01:12:44 smithi157 systemd[1]: Started Ceph mgr.z for 9efbfa8e-1162-11ee-9b2b-001a4aab830c. 2023-06-23T01:12:45.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:44 smithi157 ceph-mon[116945]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:45.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:44 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:45.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:44 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:45.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:44 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:45.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:44 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-06-23T01:12:45.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:44 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-06-23T01:12:45.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:44 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-06-23T01:12:45.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:44 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:45.144 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-06-23T01:12:45.145 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-06-23T01:12:45.145 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000112716 s, 4.5 MB/s 2023-06-23T01:12:45.146 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-06-23T01:12:45.193 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:44 smithi134 ceph-mon[116146]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:45.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:44 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:45.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:44 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:45.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:44 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:45.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:44 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-06-23T01:12:45.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:44 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-06-23T01:12:45.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:44 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-06-23T01:12:45.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:44 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:45.201 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_4 2023-06-23T01:12:45.255 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-06-23T01:12:45.255 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T01:12:45.255 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 6h/6d Inode: 217372 Links: 1 2023-06-23T01:12:45.256 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T01:12:45.256 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T01:12:45.256 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2023-06-23 01:12:44.109039754 +0000 2023-06-23T01:12:45.256 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2023-06-23 01:06:30.608108570 +0000 2023-06-23T01:12:45.256 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2023-06-23 01:06:30.608108570 +0000 2023-06-23T01:12:45.256 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: - 2023-06-23T01:12:45.257 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-06-23T01:12:45.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:44 smithi120 ceph-mon[79813]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:45.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:44 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:45.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:44 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:45.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:44 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:45.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:44 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-06-23T01:12:45.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:44 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-06-23T01:12:45.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:44 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-06-23T01:12:45.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:44 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:45.318 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-06-23T01:12:45.318 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-06-23T01:12:45.319 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 9.9526e-05 s, 5.1 MB/s 2023-06-23T01:12:45.319 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-06-23T01:12:45.375 INFO:tasks.cephadm:Deploying osd.0 on smithi120 with /dev/vg_nvme/lv_4... 2023-06-23T01:12:45.375 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-06-23T01:12:46.131 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T01:12:46.193 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:45 smithi134 ceph-mon[116146]: Deploying daemon mgr.y on smithi134 2023-06-23T01:12:46.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:45 smithi120 ceph-mon[79813]: Deploying daemon mgr.y on smithi134 2023-06-23T01:12:46.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:45 smithi157 ceph-mon[116945]: Deploying daemon mgr.y on smithi134 2023-06-23T01:12:46.377 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Jun 23 01:12:45 smithi157 conmon[119249]: 2023-06-23T01:12:45.934+0000 7ff7f320f1c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-06-23T01:12:46.377 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Jun 23 01:12:46 smithi157 conmon[119249]: 2023-06-23T01:12:46.089+0000 7ff7f320f1c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-06-23T01:12:46.377 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Jun 23 01:12:46 smithi157 conmon[119249]: 2023-06-23T01:12:46.175+0000 7ff7f320f1c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-06-23T01:12:46.694 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jun 23 01:12:46 smithi134 systemd[1]: Starting Ceph mgr.y for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:12:46.744 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Jun 23 01:12:46 smithi157 conmon[119249]: 2023-06-23T01:12:46.386+0000 7ff7f320f1c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-06-23T01:12:46.744 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Jun 23 01:12:46 smithi157 conmon[119249]: 2023-06-23T01:12:46.583+0000 7ff7f320f1c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-06-23T01:12:47.076 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:46 smithi134 ceph-mon[116146]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:47.077 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jun 23 01:12:46 smithi134 ceph-mgr[117869]: -- 172.21.15.134:0/1252779400 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e62ebfe9c0 con 0x55e62ebd4800 2023-06-23T01:12:47.077 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jun 23 01:12:47 smithi134 bash[117805]: 93dec810443cb735b1d38271909f2e0a6b07d77069a6f4a18eb8326f368ce816 2023-06-23T01:12:47.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:46 smithi157 ceph-mon[116945]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:47.127 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Jun 23 01:12:46 smithi157 conmon[119249]: 2023-06-23T01:12:46.740+0000 7ff7f320f1c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-06-23T01:12:47.127 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Jun 23 01:12:46 smithi157 conmon[119249]: 2023-06-23T01:12:46.818+0000 7ff7f320f1c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-06-23T01:12:47.184 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:46 smithi120 ceph-mon[79813]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:47.443 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jun 23 01:12:47 smithi134 systemd[1]: Started Ceph mgr.y for 9efbfa8e-1162-11ee-9b2b-001a4aab830c. 2023-06-23T01:12:47.876 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Jun 23 01:12:47 smithi157 conmon[119249]: 2023-06-23T01:12:47.405+0000 7ff7f320f1c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-06-23T01:12:48.337 INFO:teuthology.orchestra.run.smithi120.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-06-23T01:12:48.338 INFO:teuthology.orchestra.run.smithi120.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-06-23T01:12:48.338 INFO:teuthology.orchestra.run.smithi120.stderr: stderr: 10+0 records in 2023-06-23T01:12:48.338 INFO:teuthology.orchestra.run.smithi120.stderr:10+0 records out 2023-06-23T01:12:48.338 INFO:teuthology.orchestra.run.smithi120.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0283641 s, 370 MB/s 2023-06-23T01:12:48.338 INFO:teuthology.orchestra.run.smithi120.stderr: stderr: 2023-06-23T01:12:48.338 INFO:teuthology.orchestra.run.smithi120.stderr:--> Zapping successful for: 2023-06-23T01:12:48.376 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Jun 23 01:12:47 smithi157 conmon[119249]: 2023-06-23T01:12:47.928+0000 7ff7f320f1c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-06-23T01:12:48.376 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Jun 23 01:12:48 smithi157 conmon[119249]: 2023-06-23T01:12:48.029+0000 7ff7f320f1c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-06-23T01:12:48.377 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Jun 23 01:12:48 smithi157 conmon[119249]: 2023-06-23T01:12:48.122+0000 7ff7f320f1c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-06-23T01:12:48.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:48 smithi157 ceph-mon[116945]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:48.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:48 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:48.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:48 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:48.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:48 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:48.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:48 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:48.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:48 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:12:48.411 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jun 23 01:12:48 smithi134 conmon[117845]: 2023-06-23T01:12:48.245+0000 7fdd4c8f71c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-06-23T01:12:48.412 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:48 smithi134 ceph-mon[116146]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:48.412 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:48 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:48.412 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:48 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:48.412 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:48 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:48.413 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:48 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:48.413 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:48 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:12:48.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:48 smithi120 ceph-mon[79813]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:48.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:48 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:48.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:48 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:48.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:48 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:48.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:48 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:48.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:48 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:12:48.693 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jun 23 01:12:48 smithi134 conmon[117845]: 2023-06-23T01:12:48.408+0000 7fdd4c8f71c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-06-23T01:12:48.694 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jun 23 01:12:48 smithi134 conmon[117845]: 2023-06-23T01:12:48.499+0000 7fdd4c8f71c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-06-23T01:12:48.798 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph orch daemon add osd smithi120:vg_nvme/lv_4 2023-06-23T01:12:49.020 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Jun 23 01:12:48 smithi157 conmon[119249]: 2023-06-23T01:12:48.751+0000 7ff7f320f1c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-06-23T01:12:49.020 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Jun 23 01:12:48 smithi157 conmon[119249]: 2023-06-23T01:12:48.926+0000 7ff7f320f1c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-06-23T01:12:49.020 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Jun 23 01:12:49 smithi157 conmon[119249]: 2023-06-23T01:12:49.016+0000 7ff7f320f1c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-06-23T01:12:49.091 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jun 23 01:12:48 smithi134 conmon[117845]: 2023-06-23T01:12:48.723+0000 7fdd4c8f71c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-06-23T01:12:49.092 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jun 23 01:12:48 smithi134 conmon[117845]: 2023-06-23T01:12:48.927+0000 7fdd4c8f71c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-06-23T01:12:49.376 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Jun 23 01:12:49 smithi157 conmon[119249]: 2023-06-23T01:12:49.147+0000 7ff7f320f1c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-06-23T01:12:49.376 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Jun 23 01:12:49 smithi157 conmon[119249]: 2023-06-23T01:12:49.237+0000 7ff7f320f1c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-06-23T01:12:49.444 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jun 23 01:12:49 smithi134 conmon[117845]: 2023-06-23T01:12:49.088+0000 7fdd4c8f71c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-06-23T01:12:49.444 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jun 23 01:12:49 smithi134 conmon[117845]: 2023-06-23T01:12:49.165+0000 7fdd4c8f71c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-06-23T01:12:49.576 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T01:12:50.137 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:49 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:50.137 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:49 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:50.137 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:49 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:50.137 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:49 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:50.138 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:49 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:50.138 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:49 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:12:50.138 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:49 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:50.138 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:49 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:50.138 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:49 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-06-23T01:12:50.138 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:49 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-06-23T01:12:50.138 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:49 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:50.193 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jun 23 01:12:49 smithi134 conmon[117845]: 2023-06-23T01:12:49.734+0000 7fdd4c8f71c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-06-23T01:12:50.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:49 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:50.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:49 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:50.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:49 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:50.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:49 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:50.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:49 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:50.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:49 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:12:50.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:49 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:50.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:49 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:50.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:49 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-06-23T01:12:50.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:49 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-06-23T01:12:50.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:49 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:50.196 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Jun 23 01:12:50 smithi157 conmon[119249]: 2023-06-23T01:12:50.106+0000 7ff7f320f1c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-06-23T01:12:50.196 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:49 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:50.196 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:49 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:50.196 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:49 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:50.197 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:49 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:50.197 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:49 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:50.197 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:49 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:12:50.197 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:49 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:50.197 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:49 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:50.197 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:49 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-06-23T01:12:50.197 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:49 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-06-23T01:12:50.197 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:49 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:50.522 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Jun 23 01:12:50 smithi157 conmon[119249]: 2023-06-23T01:12:50.191+0000 7ff7f320f1c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-06-23T01:12:50.523 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Jun 23 01:12:50 smithi157 conmon[119249]: 2023-06-23T01:12:50.277+0000 7ff7f320f1c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-06-23T01:12:50.523 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Jun 23 01:12:50 smithi157 conmon[119249]: 2023-06-23T01:12:50.422+0000 7ff7f320f1c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-06-23T01:12:50.693 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jun 23 01:12:50 smithi134 conmon[117845]: 2023-06-23T01:12:50.212+0000 7fdd4c8f71c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-06-23T01:12:50.694 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jun 23 01:12:50 smithi134 conmon[117845]: 2023-06-23T01:12:50.300+0000 7fdd4c8f71c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-06-23T01:12:50.694 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jun 23 01:12:50 smithi134 conmon[117845]: 2023-06-23T01:12:50.391+0000 7fdd4c8f71c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-06-23T01:12:51.176 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Jun 23 01:12:50 smithi157 conmon[119249]: 2023-06-23T01:12:50.518+0000 7ff7f320f1c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-06-23T01:12:51.176 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Jun 23 01:12:50 smithi157 conmon[119249]: 2023-06-23T01:12:50.912+0000 7ff7f320f1c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-06-23T01:12:51.177 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Jun 23 01:12:51 smithi157 conmon[119249]: 2023-06-23T01:12:50.999+0000 7ff7f320f1c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-06-23T01:12:51.177 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Jun 23 01:12:51 smithi157 conmon[119249]: 2023-06-23T01:12:51.085+0000 7ff7f320f1c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-06-23T01:12:51.177 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:50 smithi157 ceph-mon[116945]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:51.177 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:50 smithi157 ceph-mon[116945]: Reconfiguring mgr.x (unknown last config time)... 2023-06-23T01:12:51.178 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:50 smithi157 ceph-mon[116945]: Reconfiguring daemon mgr.x on smithi120 2023-06-23T01:12:51.178 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:50 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:51.178 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:50 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:51.178 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:50 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:12:51.178 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:50 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:51.178 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:50 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:12:51.178 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:50 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:51.194 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jun 23 01:12:50 smithi134 conmon[117845]: 2023-06-23T01:12:50.974+0000 7fdd4c8f71c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-06-23T01:12:51.194 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jun 23 01:12:51 smithi134 conmon[117845]: 2023-06-23T01:12:51.139+0000 7fdd4c8f71c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-06-23T01:12:51.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:50 smithi134 ceph-mon[116146]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:51.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:50 smithi134 ceph-mon[116146]: Reconfiguring mgr.x (unknown last config time)... 2023-06-23T01:12:51.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:50 smithi134 ceph-mon[116146]: Reconfiguring daemon mgr.x on smithi120 2023-06-23T01:12:51.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:50 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:51.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:50 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:51.196 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:50 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:12:51.196 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:50 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:51.196 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:50 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:12:51.196 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:50 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:51.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:50 smithi120 ceph-mon[79813]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:51.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:50 smithi120 ceph-mon[79813]: Reconfiguring mgr.x (unknown last config time)... 2023-06-23T01:12:51.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:50 smithi120 ceph-mon[79813]: Reconfiguring daemon mgr.x on smithi120 2023-06-23T01:12:51.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:50 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:51.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:50 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:51.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:50 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:12:51.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:50 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:51.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:50 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:12:51.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:50 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:12:51.592 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.588+0000 7f01ee9db700 1 -- 172.21.15.120:0/3879585035 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01e8104aa0 msgr2=0x7f01e8106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:12:51.593 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.588+0000 7f01ee9db700 1 --2- 172.21.15.120:0/3879585035 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01e8104aa0 0x7f01e8106f20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f01e0009d60 tx=0x7f01e0004d10 comp rx=0 tx=0).stop 2023-06-23T01:12:51.593 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.588+0000 7f01ee9db700 1 -- 172.21.15.120:0/3879585035 shutdown_connections 2023-06-23T01:12:51.593 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.588+0000 7f01ee9db700 1 --2- 172.21.15.120:0/3879585035 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01e8104aa0 0x7f01e8106f20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:51.593 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.588+0000 7f01ee9db700 1 --2- 172.21.15.120:0/3879585035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01e80719b0 0x7f01e806fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:51.593 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.588+0000 7f01ee9db700 1 --2- 172.21.15.120:0/3879585035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01e8070360 0x7f01e80707b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:51.593 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.588+0000 7f01ee9db700 1 -- 172.21.15.120:0/3879585035 >> 172.21.15.120:0/3879585035 conn(0x7f01e80f8320 msgr2=0x7f01e80fa790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:12:51.593 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.588+0000 7f01ee9db700 1 -- 172.21.15.120:0/3879585035 shutdown_connections 2023-06-23T01:12:51.594 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.588+0000 7f01ee9db700 1 -- 172.21.15.120:0/3879585035 wait complete. 2023-06-23T01:12:51.595 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.589+0000 7f01ee9db700 1 Processor -- start 2023-06-23T01:12:51.595 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.589+0000 7f01ee9db700 1 -- start start 2023-06-23T01:12:51.595 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.590+0000 7f01ee9db700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01e8070360 0x7f01e810f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:12:51.596 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.590+0000 7f01ee9db700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01e80719b0 0x7f01e810fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:12:51.596 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.590+0000 7f01ee9db700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01e8104aa0 0x7f01e8116e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:12:51.596 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.590+0000 7f01ee9db700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e8117430 con 0x7f01e80719b0 2023-06-23T01:12:51.596 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.590+0000 7f01ee9db700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e81175a0 con 0x7f01e8104aa0 2023-06-23T01:12:51.596 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.590+0000 7f01ee9db700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e8117760 con 0x7f01e8070360 2023-06-23T01:12:51.596 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.590+0000 7f01e77fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01e80719b0 0x7f01e810fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:12:51.596 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.590+0000 7f01e77fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01e80719b0 0x7f01e810fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42362/0 (socket says 172.21.15.120:42362) 2023-06-23T01:12:51.597 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.590+0000 7f01ecf78700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01e8104aa0 0x7f01e8116e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:12:51.597 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.590+0000 7f01e77fe700 1 -- 172.21.15.120:0/712498515 learned_addr learned my addr 172.21.15.120:0/712498515 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:12:51.597 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.590+0000 7f01e7fff700 1 --2- 172.21.15.120:0/712498515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01e8070360 0x7f01e810f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:12:51.597 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.590+0000 7f01e77fe700 1 -- 172.21.15.120:0/712498515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01e8070360 msgr2=0x7f01e810f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:12:51.597 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.590+0000 7f01e77fe700 1 --2- 172.21.15.120:0/712498515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01e8070360 0x7f01e810f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:51.597 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.590+0000 7f01e77fe700 1 -- 172.21.15.120:0/712498515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01e8104aa0 msgr2=0x7f01e8116e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:12:51.597 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.590+0000 7f01e77fe700 1 --2- 172.21.15.120:0/712498515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01e8104aa0 0x7f01e8116e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:12:51.597 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.590+0000 7f01e77fe700 1 -- 172.21.15.120:0/712498515 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e0009940 con 0x7f01e80719b0 2023-06-23T01:12:51.598 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.590+0000 7f01ecf78700 1 --2- 172.21.15.120:0/712498515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01e8104aa0 0x7f01e8116e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T01:12:51.598 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.590+0000 7f01e77fe700 1 --2- 172.21.15.120:0/712498515 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01e80719b0 0x7f01e810fd90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f01dc00ebc0 tx=0x7f01dc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:12:51.598 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.591+0000 7f01e4ff9700 1 -- 172.21.15.120:0/712498515 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01dc007cc0 con 0x7f01e80719b0 2023-06-23T01:12:51.598 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.591+0000 7f01e4ff9700 1 -- 172.21.15.120:0/712498515 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01dc00f040 con 0x7f01e80719b0 2023-06-23T01:12:51.598 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.591+0000 7f01ee9db700 1 -- 172.21.15.120:0/712498515 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01e81179e0 con 0x7f01e80719b0 2023-06-23T01:12:51.598 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.591+0000 7f01e4ff9700 1 -- 172.21.15.120:0/712498515 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01dc00c880 con 0x7f01e80719b0 2023-06-23T01:12:51.598 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.591+0000 7f01ee9db700 1 -- 172.21.15.120:0/712498515 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e8117f50 con 0x7f01e80719b0 2023-06-23T01:12:51.599 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.592+0000 7f01e4ff9700 1 -- 172.21.15.120:0/712498515 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 13) v1 ==== 43276+0+0 (secure 0 0 0) 0x7f01dc00ccb0 con 0x7f01e80719b0 2023-06-23T01:12:51.599 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.592+0000 7f01ee9db700 1 -- 172.21.15.120:0/712498515 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01e81031f0 con 0x7f01e80719b0 2023-06-23T01:12:51.600 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.592+0000 7f01e4ff9700 1 --2- 172.21.15.120:0/712498515 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f01d0036890 0x7f01d0038d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:12:51.600 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.592+0000 7f01e4ff9700 1 -- 172.21.15.120:0/712498515 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f01dc048040 con 0x7f01e80719b0 2023-06-23T01:12:51.600 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.593+0000 7f01e7fff700 1 --2- 172.21.15.120:0/712498515 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f01d0036890 0x7f01d0038d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:12:51.600 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.593+0000 7f01e7fff700 1 --2- 172.21.15.120:0/712498515 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f01d0036890 0x7f01d0038d40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f01d8004f40 tx=0x7f01d800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:12:51.601 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.598+0000 7f01e4ff9700 1 -- 172.21.15.120:0/712498515 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f01dc012340 con 0x7f01e80719b0 2023-06-23T01:12:51.626 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Jun 23 01:12:51 smithi157 conmon[119249]: 2023-06-23T01:12:51.172+0000 7ff7f320f1c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-06-23T01:12:51.693 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jun 23 01:12:51 smithi134 conmon[117845]: 2023-06-23T01:12:51.228+0000 7fdd4c8f71c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-06-23T01:12:51.694 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jun 23 01:12:51 smithi134 conmon[117845]: 2023-06-23T01:12:51.352+0000 7fdd4c8f71c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-06-23T01:12:51.694 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jun 23 01:12:51 smithi134 conmon[117845]: 2023-06-23T01:12:51.441+0000 7fdd4c8f71c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-06-23T01:12:51.802 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.798+0000 7f01ee9db700 1 -- 172.21.15.120:0/712498515 --> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f01e802b1c0 con 0x7f01d0036890 2023-06-23T01:12:51.914 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:51.909+0000 7f01e4ff9700 1 -- 172.21.15.120:0/712498515 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mgrmap(e 14) v1 ==== 86211+0+0 (secure 0 0 0) 0x7f01dc022170 con 0x7f01e80719b0 2023-06-23T01:12:52.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:51 smithi134 ceph-mon[116146]: Standby manager daemon z started 2023-06-23T01:12:52.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:51 smithi134 ceph-mon[116146]: from='mgr.? 172.21.15.157:0/742384500' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-06-23T01:12:52.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:51 smithi134 ceph-mon[116146]: from='mgr.? 172.21.15.157:0/742384500' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-06-23T01:12:52.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:51 smithi134 ceph-mon[116146]: from='mgr.? 172.21.15.157:0/742384500' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-06-23T01:12:52.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:51 smithi134 ceph-mon[116146]: from='mgr.? 172.21.15.157:0/742384500' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-06-23T01:12:52.196 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:51 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T01:12:52.196 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:51 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T01:12:52.196 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:51 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:52.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:51 smithi120 ceph-mon[79813]: Standby manager daemon z started 2023-06-23T01:12:52.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:51 smithi120 ceph-mon[79813]: from='mgr.? 172.21.15.157:0/742384500' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-06-23T01:12:52.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:51 smithi120 ceph-mon[79813]: from='mgr.? 172.21.15.157:0/742384500' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-06-23T01:12:52.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:51 smithi120 ceph-mon[79813]: from='mgr.? 172.21.15.157:0/742384500' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-06-23T01:12:52.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:51 smithi120 ceph-mon[79813]: from='mgr.? 172.21.15.157:0/742384500' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-06-23T01:12:52.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:51 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T01:12:52.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:51 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T01:12:52.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:51 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:52.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:51 smithi157 ceph-mon[116945]: Standby manager daemon z started 2023-06-23T01:12:52.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:51 smithi157 ceph-mon[116945]: from='mgr.? 172.21.15.157:0/742384500' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-06-23T01:12:52.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:51 smithi157 ceph-mon[116945]: from='mgr.? 172.21.15.157:0/742384500' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-06-23T01:12:52.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:51 smithi157 ceph-mon[116945]: from='mgr.? 172.21.15.157:0/742384500' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-06-23T01:12:52.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:51 smithi157 ceph-mon[116945]: from='mgr.? 172.21.15.157:0/742384500' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-06-23T01:12:52.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:51 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T01:12:52.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:51 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T01:12:52.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:51 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:12:52.619 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jun 23 01:12:52 smithi134 conmon[117845]: 2023-06-23T01:12:52.305+0000 7fdd4c8f71c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-06-23T01:12:52.620 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jun 23 01:12:52 smithi134 conmon[117845]: 2023-06-23T01:12:52.389+0000 7fdd4c8f71c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-06-23T01:12:52.620 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jun 23 01:12:52 smithi134 conmon[117845]: 2023-06-23T01:12:52.474+0000 7fdd4c8f71c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-06-23T01:12:52.620 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jun 23 01:12:52 smithi134 conmon[117845]: 2023-06-23T01:12:52.616+0000 7fdd4c8f71c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-06-23T01:12:52.912 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jun 23 01:12:52 smithi134 conmon[117845]: 2023-06-23T01:12:52.711+0000 7fdd4c8f71c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-06-23T01:12:53.187 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jun 23 01:12:53 smithi134 conmon[117845]: 2023-06-23T01:12:53.097+0000 7fdd4c8f71c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-06-23T01:12:53.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:52 smithi134 ceph-mon[116146]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:53.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:52 smithi134 ceph-mon[116146]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:12:53.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:52 smithi134 ceph-mon[116146]: mgrmap e14: x(active, since 72s), standbys: z 2023-06-23T01:12:53.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:52 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-06-23T01:12:53.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:52 smithi120 ceph-mon[79813]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:53.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:52 smithi120 ceph-mon[79813]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:12:53.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:52 smithi120 ceph-mon[79813]: mgrmap e14: x(active, since 72s), standbys: z 2023-06-23T01:12:53.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:52 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-06-23T01:12:53.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:52 smithi157 ceph-mon[116945]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:53.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:52 smithi157 ceph-mon[116945]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:12:53.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:52 smithi157 ceph-mon[116945]: mgrmap e14: x(active, since 72s), standbys: z 2023-06-23T01:12:53.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:52 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-06-23T01:12:53.444 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jun 23 01:12:53 smithi134 conmon[117845]: 2023-06-23T01:12:53.183+0000 7fdd4c8f71c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-06-23T01:12:53.444 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jun 23 01:12:53 smithi134 conmon[117845]: 2023-06-23T01:12:53.270+0000 7fdd4c8f71c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-06-23T01:12:53.444 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jun 23 01:12:53 smithi134 conmon[117845]: 2023-06-23T01:12:53.356+0000 7fdd4c8f71c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-06-23T01:12:53.926 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:12:53.921+0000 7f01e4ff9700 1 -- 172.21.15.120:0/712498515 <== mon.0 v2:172.21.15.120:3300/0 8 ==== mgrmap(e 15) v1 ==== 129146+0+0 (secure 0 0 0) 0x7f01dc009070 con 0x7f01e80719b0 2023-06-23T01:12:54.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:53 smithi134 ceph-mon[116146]: Standby manager daemon y started 2023-06-23T01:12:54.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:53 smithi134 ceph-mon[116146]: from='mgr.? 172.21.15.134:0/1904953627' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-06-23T01:12:54.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:53 smithi134 ceph-mon[116146]: from='mgr.? 172.21.15.134:0/1904953627' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-06-23T01:12:54.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:53 smithi134 ceph-mon[116146]: from='mgr.? 172.21.15.134:0/1904953627' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-06-23T01:12:54.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:53 smithi134 ceph-mon[116146]: from='mgr.? 172.21.15.134:0/1904953627' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-06-23T01:12:54.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:53 smithi120 ceph-mon[79813]: Standby manager daemon y started 2023-06-23T01:12:54.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:53 smithi120 ceph-mon[79813]: from='mgr.? 172.21.15.134:0/1904953627' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-06-23T01:12:54.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:53 smithi120 ceph-mon[79813]: from='mgr.? 172.21.15.134:0/1904953627' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-06-23T01:12:54.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:53 smithi120 ceph-mon[79813]: from='mgr.? 172.21.15.134:0/1904953627' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-06-23T01:12:54.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:53 smithi120 ceph-mon[79813]: from='mgr.? 172.21.15.134:0/1904953627' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-06-23T01:12:54.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:53 smithi157 ceph-mon[116945]: Standby manager daemon y started 2023-06-23T01:12:54.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:53 smithi157 ceph-mon[116945]: from='mgr.? 172.21.15.134:0/1904953627' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-06-23T01:12:54.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:53 smithi157 ceph-mon[116945]: from='mgr.? 172.21.15.134:0/1904953627' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-06-23T01:12:54.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:53 smithi157 ceph-mon[116945]: from='mgr.? 172.21.15.134:0/1904953627' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-06-23T01:12:54.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:53 smithi157 ceph-mon[116945]: from='mgr.? 172.21.15.134:0/1904953627' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-06-23T01:12:55.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:54 smithi134 ceph-mon[116146]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:55.251 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:54 smithi134 ceph-mon[116146]: mgrmap e15: x(active, since 74s), standbys: z, y 2023-06-23T01:12:55.252 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:54 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-06-23T01:12:55.252 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:54 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/929594661' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3729642b-e74f-4253-bde8-535d8bfe8c86"}]: dispatch 2023-06-23T01:12:55.252 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:54 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/929594661' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3729642b-e74f-4253-bde8-535d8bfe8c86"}]': finished 2023-06-23T01:12:55.252 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:54 smithi134 ceph-mon[116146]: osdmap e5: 1 total, 0 up, 1 in 2023-06-23T01:12:55.252 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:54 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-23T01:12:55.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:54 smithi120 ceph-mon[79813]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:55.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:54 smithi120 ceph-mon[79813]: mgrmap e15: x(active, since 74s), standbys: z, y 2023-06-23T01:12:55.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:54 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-06-23T01:12:55.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:54 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/929594661' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3729642b-e74f-4253-bde8-535d8bfe8c86"}]: dispatch 2023-06-23T01:12:55.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:54 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/929594661' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3729642b-e74f-4253-bde8-535d8bfe8c86"}]': finished 2023-06-23T01:12:55.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:54 smithi120 ceph-mon[79813]: osdmap e5: 1 total, 0 up, 1 in 2023-06-23T01:12:55.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:54 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-23T01:12:55.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:54 smithi157 ceph-mon[116945]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:55.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:54 smithi157 ceph-mon[116945]: mgrmap e15: x(active, since 74s), standbys: z, y 2023-06-23T01:12:55.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:54 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-06-23T01:12:55.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:54 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/929594661' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3729642b-e74f-4253-bde8-535d8bfe8c86"}]: dispatch 2023-06-23T01:12:55.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:54 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/929594661' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3729642b-e74f-4253-bde8-535d8bfe8c86"}]': finished 2023-06-23T01:12:55.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:54 smithi157 ceph-mon[116945]: osdmap e5: 1 total, 0 up, 1 in 2023-06-23T01:12:55.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:54 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-23T01:12:56.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:55 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/2505618184' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T01:12:56.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:55 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/2505618184' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T01:12:56.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:55 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/2505618184' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T01:12:57.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:56 smithi120 ceph-mon[79813]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:57.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:56 smithi157 ceph-mon[116945]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:57.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:56 smithi134 ceph-mon[116146]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:58.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:12:57 smithi120 ceph-mon[79813]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:58.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:12:57 smithi157 ceph-mon[116945]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:12:58.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:12:57 smithi134 ceph-mon[116146]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:13:00.337 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:00 smithi120 ceph-mon[79813]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:13:00.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:00 smithi157 ceph-mon[116945]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:13:00.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:00 smithi134 ceph-mon[116146]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:13:03.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:02 smithi134 ceph-mon[116146]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:13:03.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:02 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-06-23T01:13:03.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:02 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:13:03.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:02 smithi120 ceph-mon[79813]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:13:03.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:02 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-06-23T01:13:03.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:02 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:13:03.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:02 smithi157 ceph-mon[116945]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:13:03.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:02 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-06-23T01:13:03.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:02 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:13:04.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:03 smithi134 ceph-mon[116146]: Deploying daemon osd.0 on smithi120 2023-06-23T01:13:04.259 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:03 smithi120 ceph-mon[79813]: Deploying daemon osd.0 on smithi120 2023-06-23T01:13:04.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:03 smithi157 ceph-mon[116945]: Deploying daemon osd.0 on smithi120 2023-06-23T01:13:05.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:04 smithi120 ceph-mon[79813]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:13:05.193 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:04 smithi134 ceph-mon[116146]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:13:05.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:04 smithi157 ceph-mon[116945]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:13:07.193 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:06 smithi134 ceph-mon[116146]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:13:07.250 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:06 smithi120 ceph-mon[79813]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:13:07.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:06 smithi157 ceph-mon[116945]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:13:09.196 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:08 smithi134 ceph-mon[116146]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:13:09.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:08 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:09.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:08 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:09.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:08 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:13:09.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:08 smithi120 ceph-mon[79813]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:13:09.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:08 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:09.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:08 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:09.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:08 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:13:09.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:08 smithi157 ceph-mon[116945]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:13:09.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:08 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:09.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:08 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:09.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:08 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:13:11.193 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:10 smithi134 ceph-mon[116146]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:13:11.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:10 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:11.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:10 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:11.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:10 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:13:11.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:10 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:13:11.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:10 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:11.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:10 smithi120 ceph-mon[79813]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:13:11.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:10 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:11.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:10 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:11.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:10 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:13:11.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:10 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:13:11.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:10 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:11.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:10 smithi157 ceph-mon[116945]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:13:11.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:10 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:11.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:10 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:11.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:10 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:13:11.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:10 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:13:11.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:10 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:12.018 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:13:12.015+0000 7f01e4ff9700 1 -- 172.21.15.120:0/712498515 <== mgr.14152 v2:172.21.15.120:6800/1770912857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f01e802b1c0 con 0x7f01d0036890 2023-06-23T01:13:12.019 INFO:teuthology.orchestra.run.smithi120.stdout:Created osd(s) 0 on host 'smithi120' 2023-06-23T01:13:12.021 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:13:12.018+0000 7f01ee9db700 1 -- 172.21.15.120:0/712498515 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f01d0036890 msgr2=0x7f01d0038d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:13:12.021 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:13:12.018+0000 7f01ee9db700 1 --2- 172.21.15.120:0/712498515 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f01d0036890 0x7f01d0038d40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f01d8004f40 tx=0x7f01d800b040 comp rx=0 tx=0).stop 2023-06-23T01:13:12.021 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:13:12.018+0000 7f01ee9db700 1 -- 172.21.15.120:0/712498515 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01e80719b0 msgr2=0x7f01e810fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:13:12.021 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:13:12.018+0000 7f01ee9db700 1 --2- 172.21.15.120:0/712498515 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01e80719b0 0x7f01e810fd90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f01dc00ebc0 tx=0x7f01dc00b040 comp rx=0 tx=0).stop 2023-06-23T01:13:12.022 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:13:12.018+0000 7f01ee9db700 1 -- 172.21.15.120:0/712498515 shutdown_connections 2023-06-23T01:13:12.022 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:13:12.018+0000 7f01ee9db700 1 --2- 172.21.15.120:0/712498515 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f01d0036890 0x7f01d0038d40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:13:12.022 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:13:12.018+0000 7f01ee9db700 1 --2- 172.21.15.120:0/712498515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01e8104aa0 0x7f01e8116e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:13:12.022 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:13:12.018+0000 7f01ee9db700 1 --2- 172.21.15.120:0/712498515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01e8070360 0x7f01e810f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:13:12.022 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:13:12.018+0000 7f01ee9db700 1 --2- 172.21.15.120:0/712498515 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01e80719b0 0x7f01e810fd90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:13:12.022 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:13:12.018+0000 7f01ee9db700 1 -- 172.21.15.120:0/712498515 >> 172.21.15.120:0/712498515 conn(0x7f01e80f8320 msgr2=0x7f01e8106b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:13:12.022 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:13:12.018+0000 7f01ee9db700 1 -- 172.21.15.120:0/712498515 shutdown_connections 2023-06-23T01:13:12.023 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:13:12.018+0000 7f01ee9db700 1 -- 172.21.15.120:0/712498515 wait complete. 2023-06-23T01:13:12.457 DEBUG:teuthology.orchestra.run.smithi120:osd.0> sudo journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@osd.0.service 2023-06-23T01:13:12.463 INFO:tasks.cephadm:Deploying osd.1 on smithi134 with /dev/vg_nvme/lv_4... 2023-06-23T01:13:12.463 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-06-23T01:13:12.526 INFO:journalctl@ceph.osd.0.smithi120.stdout:-- Logs begin at Fri 2023-06-23 00:53:53 UTC. -- 2023-06-23T01:13:12.925 INFO:journalctl@ceph.osd.0.smithi120.stdout:Jun 23 01:13:12 smithi120 conmon[96119]: 2023-06-23T01:13:12.592+0000 7fdc350d9540 -1 osd.0 0 log_to_monitors true 2023-06-23T01:13:13.052 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.b/config 2023-06-23T01:13:13.193 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:12 smithi134 ceph-mon[116146]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:13:13.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:12 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:13.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:12 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:13.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:12 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:13:13.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:12 smithi134 ceph-mon[116146]: from='osd.0 [v2:172.21.15.120:6802/3479894866,v1:172.21.15.120:6803/3479894866]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-06-23T01:13:13.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:12 smithi134 ceph-mon[116146]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-06-23T01:13:13.213 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:12 smithi120 ceph-mon[79813]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:13:13.213 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:12 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:13.213 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:12 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:13.213 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:12 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:13:13.214 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:12 smithi120 ceph-mon[79813]: from='osd.0 [v2:172.21.15.120:6802/3479894866,v1:172.21.15.120:6803/3479894866]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-06-23T01:13:13.214 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:12 smithi120 ceph-mon[79813]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-06-23T01:13:13.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:12 smithi157 ceph-mon[116945]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:13:13.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:12 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:13.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:12 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:13.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:12 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:13:13.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:12 smithi157 ceph-mon[116945]: from='osd.0 [v2:172.21.15.120:6802/3479894866,v1:172.21.15.120:6803/3479894866]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-06-23T01:13:13.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:12 smithi157 ceph-mon[116945]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-06-23T01:13:14.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:13 smithi120 ceph-mon[79813]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:13:14.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:13 smithi120 ceph-mon[79813]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-06-23T01:13:14.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:13 smithi120 ceph-mon[79813]: osdmap e6: 1 total, 0 up, 1 in 2023-06-23T01:13:14.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:13 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-23T01:13:14.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:13 smithi120 ceph-mon[79813]: from='osd.0 [v2:172.21.15.120:6802/3479894866,v1:172.21.15.120:6803/3479894866]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2023-06-23T01:13:14.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:13 smithi120 ceph-mon[79813]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2023-06-23T01:13:14.262 INFO:journalctl@ceph.osd.0.smithi120.stdout:Jun 23 01:13:14 smithi120 conmon[96119]: 2023-06-23T01:13:14.003+0000 7fdc2b8be700 -1 osd.0 0 waiting for initial osdmap 2023-06-23T01:13:14.262 INFO:journalctl@ceph.osd.0.smithi120.stdout:Jun 23 01:13:14 smithi120 conmon[96119]: 2023-06-23T01:13:14.007+0000 7fdc276bb700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T01:13:14.300 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:13 smithi134 ceph-mon[116146]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:13:14.300 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:13 smithi134 ceph-mon[116146]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-06-23T01:13:14.301 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:13 smithi134 ceph-mon[116146]: osdmap e6: 1 total, 0 up, 1 in 2023-06-23T01:13:14.301 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:13 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-23T01:13:14.301 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:13 smithi134 ceph-mon[116146]: from='osd.0 [v2:172.21.15.120:6802/3479894866,v1:172.21.15.120:6803/3479894866]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2023-06-23T01:13:14.301 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:13 smithi134 ceph-mon[116146]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2023-06-23T01:13:14.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:13 smithi157 ceph-mon[116945]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:13:14.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:13 smithi157 ceph-mon[116945]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-06-23T01:13:14.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:13 smithi157 ceph-mon[116945]: osdmap e6: 1 total, 0 up, 1 in 2023-06-23T01:13:14.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:13 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-23T01:13:14.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:13 smithi157 ceph-mon[116945]: from='osd.0 [v2:172.21.15.120:6802/3479894866,v1:172.21.15.120:6803/3479894866]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2023-06-23T01:13:14.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:13 smithi157 ceph-mon[116945]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2023-06-23T01:13:14.717 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-06-23T01:13:14.718 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-06-23T01:13:14.718 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2023-06-23T01:13:14.718 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2023-06-23T01:13:14.718 INFO:teuthology.orchestra.run.smithi134.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0108715 s, 965 MB/s 2023-06-23T01:13:14.718 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 2023-06-23T01:13:14.719 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2023-06-23T01:13:15.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:15 smithi120 ceph-mon[79813]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2023-06-23T01:13:15.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:15 smithi120 ceph-mon[79813]: osdmap e7: 1 total, 0 up, 1 in 2023-06-23T01:13:15.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:15 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-23T01:13:15.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:15 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-23T01:13:15.308 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:15 smithi134 ceph-mon[116146]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2023-06-23T01:13:15.308 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:15 smithi134 ceph-mon[116146]: osdmap e7: 1 total, 0 up, 1 in 2023-06-23T01:13:15.308 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:15 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-23T01:13:15.308 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:15 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-23T01:13:15.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:15 smithi157 ceph-mon[116945]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2023-06-23T01:13:15.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:15 smithi157 ceph-mon[116945]: osdmap e7: 1 total, 0 up, 1 in 2023-06-23T01:13:15.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:15 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-23T01:13:15.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:15 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-23T01:13:15.378 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph orch daemon add osd smithi134:vg_nvme/lv_4 2023-06-23T01:13:15.954 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.b/config 2023-06-23T01:13:16.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:16 smithi120 ceph-mon[79813]: purged_snaps scrub starts 2023-06-23T01:13:16.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:16 smithi120 ceph-mon[79813]: purged_snaps scrub ok 2023-06-23T01:13:16.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:16 smithi120 ceph-mon[79813]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:13:16.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:16 smithi120 ceph-mon[79813]: osd.0 [v2:172.21.15.120:6802/3479894866,v1:172.21.15.120:6803/3479894866] boot 2023-06-23T01:13:16.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:16 smithi120 ceph-mon[79813]: osdmap e8: 1 total, 1 up, 1 in 2023-06-23T01:13:16.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:16 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-23T01:13:16.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:16 smithi120 ceph-mon[79813]: Detected new or changed devices on smithi120 2023-06-23T01:13:16.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:16 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:16.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:16 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:16.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:16 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:13:16.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:16 smithi120 ceph-mon[79813]: Adjusting osd_memory_target on smithi120 to 17267M 2023-06-23T01:13:16.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:16 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:16.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:16 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:13:16.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:16 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:13:16.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:16 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:16.310 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:16 smithi134 ceph-mon[116146]: purged_snaps scrub starts 2023-06-23T01:13:16.311 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:16 smithi134 ceph-mon[116146]: purged_snaps scrub ok 2023-06-23T01:13:16.311 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:16 smithi134 ceph-mon[116146]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:13:16.311 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:16 smithi134 ceph-mon[116146]: osd.0 [v2:172.21.15.120:6802/3479894866,v1:172.21.15.120:6803/3479894866] boot 2023-06-23T01:13:16.312 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:16 smithi134 ceph-mon[116146]: osdmap e8: 1 total, 1 up, 1 in 2023-06-23T01:13:16.312 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:16 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-23T01:13:16.312 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:16 smithi134 ceph-mon[116146]: Detected new or changed devices on smithi120 2023-06-23T01:13:16.312 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:16 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:16.312 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:16 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:16.312 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:16 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:13:16.313 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:16 smithi134 ceph-mon[116146]: Adjusting osd_memory_target on smithi120 to 17267M 2023-06-23T01:13:16.313 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:16 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:16.313 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:16 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:13:16.313 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:16 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:13:16.313 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:16 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:16.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:16 smithi157 ceph-mon[116945]: purged_snaps scrub starts 2023-06-23T01:13:16.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:16 smithi157 ceph-mon[116945]: purged_snaps scrub ok 2023-06-23T01:13:16.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:16 smithi157 ceph-mon[116945]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T01:13:16.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:16 smithi157 ceph-mon[116945]: osd.0 [v2:172.21.15.120:6802/3479894866,v1:172.21.15.120:6803/3479894866] boot 2023-06-23T01:13:16.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:16 smithi157 ceph-mon[116945]: osdmap e8: 1 total, 1 up, 1 in 2023-06-23T01:13:16.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:16 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-23T01:13:16.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:16 smithi157 ceph-mon[116945]: Detected new or changed devices on smithi120 2023-06-23T01:13:16.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:16 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:16.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:16 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:16.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:16 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:13:16.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:16 smithi157 ceph-mon[116945]: Adjusting osd_memory_target on smithi120 to 17267M 2023-06-23T01:13:16.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:16 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:16.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:16 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:13:16.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:16 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:13:16.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:16 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:17.293 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.290+0000 7f357c640700 1 -- 172.21.15.134:0/389125678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35740ffde0 msgr2=0x7f35741021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:13:17.293 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.290+0000 7f357c640700 1 --2- 172.21.15.134:0/389125678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35740ffde0 0x7f35741021e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f35640049c0 tx=0x7f3564009e30 comp rx=0 tx=0).stop 2023-06-23T01:13:17.293 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.290+0000 7f357c640700 1 -- 172.21.15.134:0/389125678 shutdown_connections 2023-06-23T01:13:17.293 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.290+0000 7f357c640700 1 --2- 172.21.15.134:0/389125678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3574102870 0x7f3574104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:13:17.294 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.290+0000 7f357c640700 1 --2- 172.21.15.134:0/389125678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3574066c80 0x7f35740ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:13:17.294 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.290+0000 7f357c640700 1 --2- 172.21.15.134:0/389125678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35740ffde0 0x7f35741021e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:13:17.294 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.290+0000 7f357c640700 1 -- 172.21.15.134:0/389125678 >> 172.21.15.134:0/389125678 conn(0x7f35740f7220 msgr2=0x7f35740f9690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:13:17.294 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.290+0000 7f357c640700 1 -- 172.21.15.134:0/389125678 shutdown_connections 2023-06-23T01:13:17.294 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.290+0000 7f357c640700 1 -- 172.21.15.134:0/389125678 wait complete. 2023-06-23T01:13:17.294 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.290+0000 7f357c640700 1 Processor -- start 2023-06-23T01:13:17.294 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.291+0000 7f357c640700 1 -- start start 2023-06-23T01:13:17.296 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.291+0000 7f357c640700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3574066c80 0x7f357410d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:13:17.296 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.291+0000 7f357c640700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35740ffde0 0x7f357410de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:13:17.296 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.291+0000 7f357c640700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3574102870 0x7f3574114ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:13:17.296 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.291+0000 7f357c640700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3574115500 con 0x7f35740ffde0 2023-06-23T01:13:17.297 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.291+0000 7f357c640700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3574115670 con 0x7f3574102870 2023-06-23T01:13:17.297 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.291+0000 7f357c640700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3574115830 con 0x7f3574066c80 2023-06-23T01:13:17.297 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.291+0000 7f357abdd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3574102870 0x7f3574114ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:13:17.297 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.291+0000 7f357abdd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3574102870 0x7f3574114ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:43104/0 (socket says 172.21.15.134:43104) 2023-06-23T01:13:17.297 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.291+0000 7f357abdd700 1 -- 172.21.15.134:0/2180675142 learned_addr learned my addr 172.21.15.134:0/2180675142 (peer_addr_for_me v2:172.21.15.134:0/0) 2023-06-23T01:13:17.297 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.291+0000 7f357a3dc700 1 --2- 172.21.15.134:0/2180675142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3574066c80 0x7f357410d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:13:17.298 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.291+0000 7f357abdd700 1 -- 172.21.15.134:0/2180675142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3574066c80 msgr2=0x7f357410d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:13:17.298 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.291+0000 7f357abdd700 1 --2- 172.21.15.134:0/2180675142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3574066c80 0x7f357410d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:13:17.298 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.291+0000 7f357abdd700 1 -- 172.21.15.134:0/2180675142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35740ffde0 msgr2=0x7f357410de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:13:17.298 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.291+0000 7f357abdd700 1 --2- 172.21.15.134:0/2180675142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35740ffde0 0x7f357410de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:13:17.298 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.291+0000 7f357abdd700 1 -- 172.21.15.134:0/2180675142 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3564005040 con 0x7f3574102870 2023-06-23T01:13:17.298 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.291+0000 7f357abdd700 1 --2- 172.21.15.134:0/2180675142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3574102870 0x7f3574114ef0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f356c00cb90 tx=0x7f356c007f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:13:17.298 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.291+0000 7f356affd700 1 -- 172.21.15.134:0/2180675142 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c004020 con 0x7f3574102870 2023-06-23T01:13:17.299 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.291+0000 7f356affd700 1 -- 172.21.15.134:0/2180675142 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f356c004180 con 0x7f3574102870 2023-06-23T01:13:17.299 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.291+0000 7f356affd700 1 -- 172.21.15.134:0/2180675142 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c003a10 con 0x7f3574102870 2023-06-23T01:13:17.299 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.291+0000 7f357c640700 1 -- 172.21.15.134:0/2180675142 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3574115b10 con 0x7f3574102870 2023-06-23T01:13:17.299 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.292+0000 7f357c640700 1 -- 172.21.15.134:0/2180675142 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3574116080 con 0x7f3574102870 2023-06-23T01:13:17.299 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.292+0000 7f357c640700 1 -- 172.21.15.134:0/2180675142 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35740f8d50 con 0x7f3574102870 2023-06-23T01:13:17.300 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.294+0000 7f356affd700 1 -- 172.21.15.134:0/2180675142 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 15) v1 ==== 129146+0+0 (secure 0 0 0) 0x7f356c00a7a0 con 0x7f3574102870 2023-06-23T01:13:17.300 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.294+0000 7f356affd700 1 --2- 172.21.15.134:0/2180675142 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f356009b670 0x7f356009db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:13:17.300 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.295+0000 7f356affd700 1 -- 172.21.15.134:0/2180675142 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1763+0+0 (secure 0 0 0) 0x7f356c0c3ae0 con 0x7f3574102870 2023-06-23T01:13:17.300 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.295+0000 7f357a3dc700 1 --2- 172.21.15.134:0/2180675142 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f356009b670 0x7f356009db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:13:17.300 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.296+0000 7f357a3dc700 1 --2- 172.21.15.134:0/2180675142 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f356009b670 0x7f356009db20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3570004c40 tx=0x7f3570009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:13:17.301 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.298+0000 7f356affd700 1 -- 172.21.15.134:0/2180675142 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f356c090d10 con 0x7f3574102870 2023-06-23T01:13:17.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:17 smithi134 ceph-mon[116146]: osdmap e9: 1 total, 1 up, 1 in 2023-06-23T01:13:17.492 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:17.490+0000 7f357c640700 1 -- 172.21.15.134:0/2180675142 --> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f357402b370 con 0x7f356009b670 2023-06-23T01:13:17.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:17 smithi120 ceph-mon[79813]: osdmap e9: 1 total, 1 up, 1 in 2023-06-23T01:13:17.626 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:17 smithi157 ceph-mon[116945]: osdmap e9: 1 total, 1 up, 1 in 2023-06-23T01:13:18.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:18 smithi134 ceph-mon[116146]: pgmap v48: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-23T01:13:18.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:18 smithi134 ceph-mon[116146]: from='client.24152 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:13:18.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:18 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T01:13:18.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:18 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T01:13:18.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:18 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:13:18.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:18 smithi120 ceph-mon[79813]: pgmap v48: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-23T01:13:18.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:18 smithi120 ceph-mon[79813]: from='client.24152 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:13:18.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:18 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T01:13:18.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:18 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T01:13:18.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:18 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:13:18.626 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:18 smithi157 ceph-mon[116945]: pgmap v48: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-23T01:13:18.626 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:18 smithi157 ceph-mon[116945]: from='client.24152 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:13:18.627 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:18 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T01:13:18.627 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:18 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T01:13:18.627 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:18 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:13:20.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:20 smithi134 ceph-mon[116146]: pgmap v49: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-23T01:13:20.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:20 smithi134 ceph-mon[116146]: from='client.? 172.21.15.134:0/2706587215' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58240258-afd5-4991-9d5b-a80d365e4fc6"}]: dispatch 2023-06-23T01:13:20.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:20 smithi134 ceph-mon[116146]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58240258-afd5-4991-9d5b-a80d365e4fc6"}]: dispatch 2023-06-23T01:13:20.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:20 smithi134 ceph-mon[116146]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "58240258-afd5-4991-9d5b-a80d365e4fc6"}]': finished 2023-06-23T01:13:20.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:20 smithi134 ceph-mon[116146]: osdmap e10: 2 total, 1 up, 2 in 2023-06-23T01:13:20.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:20 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-23T01:13:20.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:20 smithi120 ceph-mon[79813]: pgmap v49: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-23T01:13:20.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:20 smithi120 ceph-mon[79813]: from='client.? 172.21.15.134:0/2706587215' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58240258-afd5-4991-9d5b-a80d365e4fc6"}]: dispatch 2023-06-23T01:13:20.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:20 smithi120 ceph-mon[79813]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58240258-afd5-4991-9d5b-a80d365e4fc6"}]: dispatch 2023-06-23T01:13:20.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:20 smithi120 ceph-mon[79813]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "58240258-afd5-4991-9d5b-a80d365e4fc6"}]': finished 2023-06-23T01:13:20.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:20 smithi120 ceph-mon[79813]: osdmap e10: 2 total, 1 up, 2 in 2023-06-23T01:13:20.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:20 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-23T01:13:20.626 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:20 smithi157 ceph-mon[116945]: pgmap v49: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-23T01:13:20.626 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:20 smithi157 ceph-mon[116945]: from='client.? 172.21.15.134:0/2706587215' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58240258-afd5-4991-9d5b-a80d365e4fc6"}]: dispatch 2023-06-23T01:13:20.627 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:20 smithi157 ceph-mon[116945]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58240258-afd5-4991-9d5b-a80d365e4fc6"}]: dispatch 2023-06-23T01:13:20.627 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:20 smithi157 ceph-mon[116945]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "58240258-afd5-4991-9d5b-a80d365e4fc6"}]': finished 2023-06-23T01:13:20.627 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:20 smithi157 ceph-mon[116945]: osdmap e10: 2 total, 1 up, 2 in 2023-06-23T01:13:20.627 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:20 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-23T01:13:21.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:21 smithi134 ceph-mon[116146]: from='client.? 172.21.15.134:0/1849745367' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T01:13:21.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:21 smithi120 ceph-mon[79813]: from='client.? 172.21.15.134:0/1849745367' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T01:13:21.626 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:21 smithi157 ceph-mon[116945]: from='client.? 172.21.15.134:0/1849745367' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T01:13:23.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:22 smithi134 ceph-mon[116146]: pgmap v51: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-23T01:13:23.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:22 smithi120 ceph-mon[79813]: pgmap v51: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-23T01:13:23.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:22 smithi157 ceph-mon[116945]: pgmap v51: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-23T01:13:25.193 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:24 smithi134 ceph-mon[116146]: pgmap v52: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-23T01:13:25.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:24 smithi120 ceph-mon[79813]: pgmap v52: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-23T01:13:25.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:24 smithi157 ceph-mon[116945]: pgmap v52: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-23T01:13:27.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:26 smithi134 ceph-mon[116146]: pgmap v53: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-23T01:13:27.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:26 smithi120 ceph-mon[79813]: pgmap v53: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-23T01:13:27.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:26 smithi157 ceph-mon[116945]: pgmap v53: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-23T01:13:28.179 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:27 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-06-23T01:13:28.179 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:27 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:13:28.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:27 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-06-23T01:13:28.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:27 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:13:28.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:27 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-06-23T01:13:28.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:27 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:13:29.156 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:28 smithi134 ceph-mon[116146]: pgmap v54: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-23T01:13:29.156 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:28 smithi134 ceph-mon[116146]: Deploying daemon osd.1 on smithi134 2023-06-23T01:13:29.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:28 smithi120 ceph-mon[79813]: pgmap v54: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-23T01:13:29.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:28 smithi120 ceph-mon[79813]: Deploying daemon osd.1 on smithi134 2023-06-23T01:13:29.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:28 smithi157 ceph-mon[116945]: pgmap v54: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-23T01:13:29.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:28 smithi157 ceph-mon[116945]: Deploying daemon osd.1 on smithi134 2023-06-23T01:13:31.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:30 smithi134 ceph-mon[116146]: pgmap v55: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-23T01:13:31.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:30 smithi120 ceph-mon[79813]: pgmap v55: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-23T01:13:31.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:30 smithi157 ceph-mon[116945]: pgmap v55: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-23T01:13:33.193 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:32 smithi134 ceph-mon[116146]: pgmap v56: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-23T01:13:33.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:32 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:13:33.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:32 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:33.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:32 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:33.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:32 smithi120 ceph-mon[79813]: pgmap v56: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-23T01:13:33.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:32 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:13:33.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:32 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:33.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:32 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:33.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:32 smithi157 ceph-mon[116945]: pgmap v56: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-23T01:13:33.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:32 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:13:33.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:32 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:33.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:32 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:35.193 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:34 smithi134 ceph-mon[116146]: pgmap v57: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-23T01:13:35.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:34 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:35.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:34 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:35.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:34 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:13:35.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:34 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:13:35.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:34 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:35.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:34 smithi120 ceph-mon[79813]: pgmap v57: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-23T01:13:35.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:34 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:35.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:34 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:35.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:34 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:13:35.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:34 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:13:35.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:34 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:35.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:34 smithi157 ceph-mon[116945]: pgmap v57: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-23T01:13:35.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:34 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:35.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:34 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:35.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:34 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:13:35.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:34 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:13:35.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:34 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:35.862 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 1 on host 'smithi134' 2023-06-23T01:13:35.862 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:35.858+0000 7f356affd700 1 -- 172.21.15.134:0/2180675142 <== mgr.14152 v2:172.21.15.120:6800/1770912857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f357402b370 con 0x7f356009b670 2023-06-23T01:13:35.864 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:35.860+0000 7f357c640700 1 -- 172.21.15.134:0/2180675142 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f356009b670 msgr2=0x7f356009db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:13:35.864 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:35.860+0000 7f357c640700 1 --2- 172.21.15.134:0/2180675142 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f356009b670 0x7f356009db20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3570004c40 tx=0x7f3570009040 comp rx=0 tx=0).stop 2023-06-23T01:13:35.864 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:35.860+0000 7f357c640700 1 -- 172.21.15.134:0/2180675142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3574102870 msgr2=0x7f3574114ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:13:35.865 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:35.860+0000 7f357c640700 1 --2- 172.21.15.134:0/2180675142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3574102870 0x7f3574114ef0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f356c00cb90 tx=0x7f356c007f50 comp rx=0 tx=0).stop 2023-06-23T01:13:35.865 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:35.860+0000 7f357c640700 1 -- 172.21.15.134:0/2180675142 shutdown_connections 2023-06-23T01:13:35.865 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:35.860+0000 7f357c640700 1 --2- 172.21.15.134:0/2180675142 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f356009b670 0x7f356009db20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:13:35.865 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:35.860+0000 7f357c640700 1 --2- 172.21.15.134:0/2180675142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3574102870 0x7f3574114ef0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:13:35.866 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:35.860+0000 7f357c640700 1 --2- 172.21.15.134:0/2180675142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3574066c80 0x7f357410d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:13:35.866 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:35.860+0000 7f357c640700 1 --2- 172.21.15.134:0/2180675142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35740ffde0 0x7f357410de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:13:35.866 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:35.860+0000 7f357c640700 1 -- 172.21.15.134:0/2180675142 >> 172.21.15.134:0/2180675142 conn(0x7f35740f7220 msgr2=0x7f35740fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:13:35.866 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:35.860+0000 7f357c640700 1 -- 172.21.15.134:0/2180675142 shutdown_connections 2023-06-23T01:13:35.867 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:13:35.860+0000 7f357c640700 1 -- 172.21.15.134:0/2180675142 wait complete. 2023-06-23T01:13:36.509 DEBUG:teuthology.orchestra.run.smithi134:osd.1> sudo journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@osd.1.service 2023-06-23T01:13:36.511 INFO:tasks.cephadm:Deploying osd.2 on smithi157 with /dev/vg_nvme/lv_4... 2023-06-23T01:13:36.512 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-06-23T01:13:36.539 INFO:journalctl@ceph.osd.1.smithi134.stdout:-- Logs begin at Fri 2023-06-23 00:49:19 UTC. -- 2023-06-23T01:13:37.099 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.c/config 2023-06-23T01:13:37.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:36 smithi157 ceph-mon[116945]: pgmap v58: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-23T01:13:37.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:36 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:37.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:36 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:37.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:36 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:13:37.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:36 smithi157 ceph-mon[116945]: from='osd.1 [v2:172.21.15.134:6800/3046662876,v1:172.21.15.134:6801/3046662876]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-06-23T01:13:37.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:36 smithi157 ceph-mon[116945]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-06-23T01:13:37.164 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:36 smithi134 ceph-mon[116146]: pgmap v58: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-23T01:13:37.165 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:36 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:37.165 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:36 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:37.165 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:36 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:13:37.165 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:36 smithi134 ceph-mon[116146]: from='osd.1 [v2:172.21.15.134:6800/3046662876,v1:172.21.15.134:6801/3046662876]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-06-23T01:13:37.165 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:36 smithi134 ceph-mon[116146]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-06-23T01:13:37.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:36 smithi120 ceph-mon[79813]: pgmap v58: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-23T01:13:37.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:36 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:37.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:36 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:37.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:36 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:13:37.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:36 smithi120 ceph-mon[79813]: from='osd.1 [v2:172.21.15.134:6800/3046662876,v1:172.21.15.134:6801/3046662876]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-06-23T01:13:37.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:36 smithi120 ceph-mon[79813]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-06-23T01:13:38.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:37 smithi134 ceph-mon[116146]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-06-23T01:13:38.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:37 smithi134 ceph-mon[116146]: osdmap e11: 2 total, 1 up, 2 in 2023-06-23T01:13:38.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:37 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-23T01:13:38.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:37 smithi134 ceph-mon[116146]: from='osd.1 [v2:172.21.15.134:6800/3046662876,v1:172.21.15.134:6801/3046662876]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-06-23T01:13:38.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:37 smithi134 ceph-mon[116146]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-06-23T01:13:38.195 INFO:journalctl@ceph.osd.1.smithi134.stdout:Jun 23 01:13:37 smithi134 conmon[122645]: 2023-06-23T01:13:37.948+0000 7fc893813700 -1 osd.1 0 waiting for initial osdmap 2023-06-23T01:13:38.195 INFO:journalctl@ceph.osd.1.smithi134.stdout:Jun 23 01:13:37 smithi134 conmon[122645]: 2023-06-23T01:13:37.952+0000 7fc88f610700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T01:13:38.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:37 smithi120 ceph-mon[79813]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-06-23T01:13:38.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:37 smithi120 ceph-mon[79813]: osdmap e11: 2 total, 1 up, 2 in 2023-06-23T01:13:38.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:37 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-23T01:13:38.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:37 smithi120 ceph-mon[79813]: from='osd.1 [v2:172.21.15.134:6800/3046662876,v1:172.21.15.134:6801/3046662876]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-06-23T01:13:38.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:37 smithi120 ceph-mon[79813]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-06-23T01:13:38.323 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:37 smithi157 ceph-mon[116945]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-06-23T01:13:38.323 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:37 smithi157 ceph-mon[116945]: osdmap e11: 2 total, 1 up, 2 in 2023-06-23T01:13:38.323 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:37 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-23T01:13:38.323 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:37 smithi157 ceph-mon[116945]: from='osd.1 [v2:172.21.15.134:6800/3046662876,v1:172.21.15.134:6801/3046662876]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-06-23T01:13:38.323 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:37 smithi157 ceph-mon[116945]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-06-23T01:13:38.814 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-06-23T01:13:38.814 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-06-23T01:13:38.815 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2023-06-23T01:13:38.815 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2023-06-23T01:13:38.815 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00797307 s, 1.3 GB/s 2023-06-23T01:13:38.816 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2023-06-23T01:13:39.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:38 smithi157 ceph-mon[116945]: pgmap v60: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-23T01:13:39.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:38 smithi157 ceph-mon[116945]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-06-23T01:13:39.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:38 smithi157 ceph-mon[116945]: osdmap e12: 2 total, 1 up, 2 in 2023-06-23T01:13:39.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:38 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-23T01:13:39.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:38 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-23T01:13:39.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:38 smithi120 ceph-mon[79813]: pgmap v60: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-23T01:13:39.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:38 smithi120 ceph-mon[79813]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-06-23T01:13:39.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:38 smithi120 ceph-mon[79813]: osdmap e12: 2 total, 1 up, 2 in 2023-06-23T01:13:39.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:38 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-23T01:13:39.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:38 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-23T01:13:39.429 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph orch daemon add osd smithi157:vg_nvme/lv_4 2023-06-23T01:13:39.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:38 smithi134 ceph-mon[116146]: pgmap v60: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-23T01:13:39.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:38 smithi134 ceph-mon[116146]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-06-23T01:13:39.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:38 smithi134 ceph-mon[116146]: osdmap e12: 2 total, 1 up, 2 in 2023-06-23T01:13:39.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:38 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-23T01:13:39.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:38 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-23T01:13:40.006 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.c/config 2023-06-23T01:13:40.220 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:39 smithi157 ceph-mon[116945]: purged_snaps scrub starts 2023-06-23T01:13:40.221 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:39 smithi157 ceph-mon[116945]: purged_snaps scrub ok 2023-06-23T01:13:40.221 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:39 smithi157 ceph-mon[116945]: osd.1 [v2:172.21.15.134:6800/3046662876,v1:172.21.15.134:6801/3046662876] boot 2023-06-23T01:13:40.221 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:39 smithi157 ceph-mon[116945]: osdmap e13: 2 total, 2 up, 2 in 2023-06-23T01:13:40.221 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:39 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-23T01:13:40.221 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:39 smithi157 ceph-mon[116945]: pgmap v63: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-23T01:13:40.221 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:39 smithi157 ceph-mon[116945]: Detected new or changed devices on smithi134 2023-06-23T01:13:40.222 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:39 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:40.222 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:39 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:40.222 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:39 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:13:40.222 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:39 smithi157 ceph-mon[116945]: Adjusting osd_memory_target on smithi134 to 17172M 2023-06-23T01:13:40.222 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:39 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:40.222 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:39 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:13:40.222 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:39 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:13:40.223 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:39 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:40.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:39 smithi120 ceph-mon[79813]: purged_snaps scrub starts 2023-06-23T01:13:40.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:39 smithi120 ceph-mon[79813]: purged_snaps scrub ok 2023-06-23T01:13:40.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:39 smithi120 ceph-mon[79813]: osd.1 [v2:172.21.15.134:6800/3046662876,v1:172.21.15.134:6801/3046662876] boot 2023-06-23T01:13:40.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:39 smithi120 ceph-mon[79813]: osdmap e13: 2 total, 2 up, 2 in 2023-06-23T01:13:40.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:39 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-23T01:13:40.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:39 smithi120 ceph-mon[79813]: pgmap v63: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-23T01:13:40.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:39 smithi120 ceph-mon[79813]: Detected new or changed devices on smithi134 2023-06-23T01:13:40.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:39 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:40.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:39 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:40.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:39 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:13:40.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:39 smithi120 ceph-mon[79813]: Adjusting osd_memory_target on smithi134 to 17172M 2023-06-23T01:13:40.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:39 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:40.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:39 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:13:40.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:39 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:13:40.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:39 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:40.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:39 smithi134 ceph-mon[116146]: purged_snaps scrub starts 2023-06-23T01:13:40.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:39 smithi134 ceph-mon[116146]: purged_snaps scrub ok 2023-06-23T01:13:40.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:39 smithi134 ceph-mon[116146]: osd.1 [v2:172.21.15.134:6800/3046662876,v1:172.21.15.134:6801/3046662876] boot 2023-06-23T01:13:40.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:39 smithi134 ceph-mon[116146]: osdmap e13: 2 total, 2 up, 2 in 2023-06-23T01:13:40.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:39 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-23T01:13:40.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:39 smithi134 ceph-mon[116146]: pgmap v63: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-23T01:13:40.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:39 smithi134 ceph-mon[116146]: Detected new or changed devices on smithi134 2023-06-23T01:13:40.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:39 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:40.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:39 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:40.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:39 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:13:40.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:39 smithi134 ceph-mon[116146]: Adjusting osd_memory_target on smithi134 to 17172M 2023-06-23T01:13:40.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:39 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:40.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:39 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:13:40.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:39 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:13:40.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:39 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:41.242 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:41 smithi157 ceph-mon[116945]: osdmap e14: 2 total, 2 up, 2 in 2023-06-23T01:13:41.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.381+0000 7f768fbea700 1 -- 172.21.15.157:0/2160342654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76880fd6a0 msgr2=0x7f7688103f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:13:41.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.381+0000 7f768fbea700 1 --2- 172.21.15.157:0/2160342654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76880fd6a0 0x7f7688103f60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7680009d60 tx=0x7f7680004d10 comp rx=0 tx=0).stop 2023-06-23T01:13:41.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.381+0000 7f768fbea700 1 -- 172.21.15.157:0/2160342654 shutdown_connections 2023-06-23T01:13:41.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.381+0000 7f768fbea700 1 --2- 172.21.15.157:0/2160342654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76880fd6a0 0x7f7688103f60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:13:41.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.381+0000 7f768fbea700 1 --2- 172.21.15.157:0/2160342654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76880fbb00 0x7f76880fbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:13:41.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.381+0000 7f768fbea700 1 --2- 172.21.15.157:0/2160342654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76880fcd10 0x7f76880fd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:13:41.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.381+0000 7f768fbea700 1 -- 172.21.15.157:0/2160342654 >> 172.21.15.157:0/2160342654 conn(0x7f76880f7220 msgr2=0x7f76880f9690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:13:41.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.381+0000 7f768fbea700 1 -- 172.21.15.157:0/2160342654 shutdown_connections 2023-06-23T01:13:41.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.381+0000 7f768fbea700 1 -- 172.21.15.157:0/2160342654 wait complete. 2023-06-23T01:13:41.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.382+0000 7f768fbea700 1 Processor -- start 2023-06-23T01:13:41.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.382+0000 7f768fbea700 1 -- start start 2023-06-23T01:13:41.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.382+0000 7f768fbea700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76880fbb00 0x7f7688066a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:13:41.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.382+0000 7f768fbea700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76880fcd10 0x7f76881125f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:13:41.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.382+0000 7f768fbea700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76880fd6a0 0x7f7688114c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:13:41.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.382+0000 7f768fbea700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7688115400 con 0x7f76880fd6a0 2023-06-23T01:13:41.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.382+0000 7f768fbea700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7688115570 con 0x7f76880fbb00 2023-06-23T01:13:41.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.382+0000 7f768fbea700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7688115730 con 0x7f76880fcd10 2023-06-23T01:13:41.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.382+0000 7f768d185700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76880fcd10 0x7f76881125f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:13:41.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.382+0000 7f768d185700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76880fcd10 0x7f76881125f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38982/0 (socket says 172.21.15.157:38982) 2023-06-23T01:13:41.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.382+0000 7f768d185700 1 -- 172.21.15.157:0/3374319509 learned_addr learned my addr 172.21.15.157:0/3374319509 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-06-23T01:13:41.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.382+0000 7f768d986700 1 --2- 172.21.15.157:0/3374319509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76880fbb00 0x7f7688066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:13:41.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.382+0000 7f768d185700 1 -- 172.21.15.157:0/3374319509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76880fbb00 msgr2=0x7f7688066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:13:41.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.383+0000 7f768d185700 1 --2- 172.21.15.157:0/3374319509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76880fbb00 0x7f7688066a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:13:41.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.383+0000 7f768d185700 1 -- 172.21.15.157:0/3374319509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76880fd6a0 msgr2=0x7f7688114c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:13:41.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.383+0000 7f768d185700 1 --2- 172.21.15.157:0/3374319509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76880fd6a0 0x7f7688114c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:13:41.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.383+0000 7f768d185700 1 -- 172.21.15.157:0/3374319509 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7680009940 con 0x7f76880fcd10 2023-06-23T01:13:41.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.383+0000 7f768d185700 1 --2- 172.21.15.157:0/3374319509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76880fcd10 0x7f76881125f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f767800b810 tx=0x7f7678009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:13:41.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.383+0000 7f767e7fc700 1 -- 172.21.15.157:0/3374319509 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767800f510 con 0x7f76880fcd10 2023-06-23T01:13:41.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.383+0000 7f767e7fc700 1 -- 172.21.15.157:0/3374319509 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f767800fb40 con 0x7f76880fcd10 2023-06-23T01:13:41.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.383+0000 7f767e7fc700 1 -- 172.21.15.157:0/3374319509 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7678011c90 con 0x7f76880fcd10 2023-06-23T01:13:41.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.383+0000 7f768fbea700 1 -- 172.21.15.157:0/3374319509 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76881159b0 con 0x7f76880fcd10 2023-06-23T01:13:41.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.383+0000 7f768fbea700 1 -- 172.21.15.157:0/3374319509 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7688115ea0 con 0x7f76880fcd10 2023-06-23T01:13:41.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.384+0000 7f768fbea700 1 -- 172.21.15.157:0/3374319509 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7688102880 con 0x7f76880fcd10 2023-06-23T01:13:41.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.385+0000 7f767e7fc700 1 -- 172.21.15.157:0/3374319509 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 15) v1 ==== 129146+0+0 (secure 0 0 0) 0x7f767800fcb0 con 0x7f76880fcd10 2023-06-23T01:13:41.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.385+0000 7f767e7fc700 1 --2- 172.21.15.157:0/3374319509 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f767409b670 0x7f767409db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:13:41.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.386+0000 7f767e7fc700 1 -- 172.21.15.157:0/3374319509 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2305+0+0 (secure 0 0 0) 0x7f76780c3db0 con 0x7f76880fcd10 2023-06-23T01:13:41.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.386+0000 7f768d986700 1 --2- 172.21.15.157:0/3374319509 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f767409b670 0x7f767409db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:13:41.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.386+0000 7f768d986700 1 --2- 172.21.15.157:0/3374319509 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f767409b670 0x7f767409db20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f76880fcb70 tx=0x7f76840091a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:13:41.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.389+0000 7f767e7fc700 1 -- 172.21.15.157:0/3374319509 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f7678090ee0 con 0x7f76880fcd10 2023-06-23T01:13:41.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:41 smithi134 ceph-mon[116146]: osdmap e14: 2 total, 2 up, 2 in 2023-06-23T01:13:41.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:41 smithi120 ceph-mon[79813]: osdmap e14: 2 total, 2 up, 2 in 2023-06-23T01:13:41.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:41.579+0000 7f768fbea700 1 -- 172.21.15.157:0/3374319509 --> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f768802b370 con 0x7f767409b670 2023-06-23T01:13:42.365 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:42 smithi157 ceph-mon[116945]: pgmap v65: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-06-23T01:13:42.365 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:42 smithi157 ceph-mon[116945]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:13:42.365 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:42 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T01:13:42.365 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:42 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T01:13:42.365 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:42 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:13:42.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:42 smithi134 ceph-mon[116146]: pgmap v65: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-06-23T01:13:42.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:42 smithi134 ceph-mon[116146]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:13:42.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:42 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T01:13:42.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:42 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T01:13:42.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:42 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:13:42.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:42 smithi120 ceph-mon[79813]: pgmap v65: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-06-23T01:13:42.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:42 smithi120 ceph-mon[79813]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:13:42.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:42 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T01:13:42.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:42 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T01:13:42.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:42 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:13:44.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:44 smithi157 ceph-mon[116945]: pgmap v66: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-06-23T01:13:44.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:44 smithi134 ceph-mon[116146]: pgmap v66: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-06-23T01:13:44.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:44 smithi120 ceph-mon[79813]: pgmap v66: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-06-23T01:13:45.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:45 smithi157 ceph-mon[116945]: from='client.? 172.21.15.157:0/2796176476' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05988a7a-7cf2-4fbb-9d4f-22604cd2f1c4"}]: dispatch 2023-06-23T01:13:45.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:45 smithi157 ceph-mon[116945]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05988a7a-7cf2-4fbb-9d4f-22604cd2f1c4"}]: dispatch 2023-06-23T01:13:45.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:45 smithi157 ceph-mon[116945]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "05988a7a-7cf2-4fbb-9d4f-22604cd2f1c4"}]': finished 2023-06-23T01:13:45.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:45 smithi157 ceph-mon[116945]: osdmap e15: 3 total, 2 up, 3 in 2023-06-23T01:13:45.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:45 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-23T01:13:45.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:45 smithi157 ceph-mon[116945]: from='client.? 172.21.15.157:0/4248793602' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T01:13:45.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:45 smithi134 ceph-mon[116146]: from='client.? 172.21.15.157:0/2796176476' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05988a7a-7cf2-4fbb-9d4f-22604cd2f1c4"}]: dispatch 2023-06-23T01:13:45.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:45 smithi134 ceph-mon[116146]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05988a7a-7cf2-4fbb-9d4f-22604cd2f1c4"}]: dispatch 2023-06-23T01:13:45.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:45 smithi134 ceph-mon[116146]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "05988a7a-7cf2-4fbb-9d4f-22604cd2f1c4"}]': finished 2023-06-23T01:13:45.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:45 smithi134 ceph-mon[116146]: osdmap e15: 3 total, 2 up, 3 in 2023-06-23T01:13:45.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:45 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-23T01:13:45.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:45 smithi134 ceph-mon[116146]: from='client.? 172.21.15.157:0/4248793602' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T01:13:45.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:45 smithi120 ceph-mon[79813]: from='client.? 172.21.15.157:0/2796176476' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05988a7a-7cf2-4fbb-9d4f-22604cd2f1c4"}]: dispatch 2023-06-23T01:13:45.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:45 smithi120 ceph-mon[79813]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05988a7a-7cf2-4fbb-9d4f-22604cd2f1c4"}]: dispatch 2023-06-23T01:13:45.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:45 smithi120 ceph-mon[79813]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "05988a7a-7cf2-4fbb-9d4f-22604cd2f1c4"}]': finished 2023-06-23T01:13:45.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:45 smithi120 ceph-mon[79813]: osdmap e15: 3 total, 2 up, 3 in 2023-06-23T01:13:45.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:45 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-23T01:13:45.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:45 smithi120 ceph-mon[79813]: from='client.? 172.21.15.157:0/4248793602' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T01:13:46.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:46 smithi120 ceph-mon[79813]: pgmap v68: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-06-23T01:13:46.626 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:46 smithi157 ceph-mon[116945]: pgmap v68: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-06-23T01:13:46.693 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:46 smithi134 ceph-mon[116146]: pgmap v68: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-06-23T01:13:49.193 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:48 smithi134 ceph-mon[116146]: pgmap v69: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-23T01:13:49.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:48 smithi120 ceph-mon[79813]: pgmap v69: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-23T01:13:49.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:48 smithi157 ceph-mon[116945]: pgmap v69: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-23T01:13:51.193 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:50 smithi134 ceph-mon[116146]: pgmap v70: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-23T01:13:51.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:50 smithi120 ceph-mon[79813]: pgmap v70: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-23T01:13:51.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:50 smithi157 ceph-mon[116945]: pgmap v70: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-23T01:13:52.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:51 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-06-23T01:13:52.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:51 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:13:52.213 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:51 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-06-23T01:13:52.213 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:51 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:13:52.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:51 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-06-23T01:13:52.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:51 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:13:53.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:52 smithi134 ceph-mon[116146]: pgmap v71: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-23T01:13:53.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:52 smithi134 ceph-mon[116146]: Deploying daemon osd.2 on smithi157 2023-06-23T01:13:53.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:52 smithi120 ceph-mon[79813]: pgmap v71: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-23T01:13:53.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:52 smithi120 ceph-mon[79813]: Deploying daemon osd.2 on smithi157 2023-06-23T01:13:53.296 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:52 smithi157 ceph-mon[116945]: pgmap v71: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-23T01:13:53.296 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:52 smithi157 ceph-mon[116945]: Deploying daemon osd.2 on smithi157 2023-06-23T01:13:55.193 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:54 smithi134 ceph-mon[116146]: pgmap v72: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-23T01:13:55.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:54 smithi120 ceph-mon[79813]: pgmap v72: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-23T01:13:55.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:54 smithi157 ceph-mon[116945]: pgmap v72: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-23T01:13:57.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:56 smithi134 ceph-mon[116146]: pgmap v73: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-23T01:13:57.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:56 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:57.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:56 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:57.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:56 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:13:57.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:56 smithi120 ceph-mon[79813]: pgmap v73: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-23T01:13:57.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:56 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:57.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:56 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:57.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:56 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:13:57.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:56 smithi157 ceph-mon[116945]: pgmap v73: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-23T01:13:57.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:56 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:57.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:56 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:13:57.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:56 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:13:59.192 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:58 smithi157 ceph-mon[116945]: pgmap v74: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-23T01:13:59.193 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:58 smithi134 ceph-mon[116146]: pgmap v74: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-23T01:13:59.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:58 smithi120 ceph-mon[79813]: pgmap v74: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-23T01:13:59.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:59.896+0000 7f767e7fc700 1 -- 172.21.15.157:0/3374319509 <== mgr.14152 v2:172.21.15.120:6800/1770912857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f768802b370 con 0x7f767409b670 2023-06-23T01:13:59.901 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 2 on host 'smithi157' 2023-06-23T01:13:59.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:59.900+0000 7f768fbea700 1 -- 172.21.15.157:0/3374319509 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f767409b670 msgr2=0x7f767409db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:13:59.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:59.900+0000 7f768fbea700 1 --2- 172.21.15.157:0/3374319509 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f767409b670 0x7f767409db20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f76880fcb70 tx=0x7f76840091a0 comp rx=0 tx=0).stop 2023-06-23T01:13:59.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:59.900+0000 7f768fbea700 1 -- 172.21.15.157:0/3374319509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76880fcd10 msgr2=0x7f76881125f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:13:59.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:59.900+0000 7f768fbea700 1 --2- 172.21.15.157:0/3374319509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76880fcd10 0x7f76881125f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f767800b810 tx=0x7f7678009250 comp rx=0 tx=0).stop 2023-06-23T01:13:59.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:59.900+0000 7f768fbea700 1 -- 172.21.15.157:0/3374319509 shutdown_connections 2023-06-23T01:13:59.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:59.900+0000 7f768fbea700 1 --2- 172.21.15.157:0/3374319509 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f767409b670 0x7f767409db20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:13:59.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:59.900+0000 7f768fbea700 1 --2- 172.21.15.157:0/3374319509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76880fd6a0 0x7f7688114c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:13:59.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:59.900+0000 7f768fbea700 1 --2- 172.21.15.157:0/3374319509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76880fbb00 0x7f7688066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:13:59.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:59.900+0000 7f768fbea700 1 --2- 172.21.15.157:0/3374319509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76880fcd10 0x7f76881125f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:13:59.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:59.900+0000 7f768fbea700 1 -- 172.21.15.157:0/3374319509 >> 172.21.15.157:0/3374319509 conn(0x7f76880f7220 msgr2=0x7f76880f9310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:13:59.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:59.900+0000 7f768fbea700 1 -- 172.21.15.157:0/3374319509 shutdown_connections 2023-06-23T01:13:59.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:13:59.900+0000 7f768fbea700 1 -- 172.21.15.157:0/3374319509 wait complete. 2023-06-23T01:14:00.121 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:59 smithi157 ceph-mon[116945]: pgmap v75: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-23T01:14:00.121 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:59 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:00.121 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:59 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:00.121 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:59 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:14:00.121 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:59 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:14:00.122 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:59 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:00.122 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:59 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:00.122 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:59 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:14:00.122 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:13:59 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:00.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:59 smithi120 ceph-mon[79813]: pgmap v75: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-23T01:14:00.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:59 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:00.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:59 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:00.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:59 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:14:00.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:59 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:14:00.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:59 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:00.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:59 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:00.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:59 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:14:00.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:13:59 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:00.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:59 smithi134 ceph-mon[116146]: pgmap v75: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-23T01:14:00.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:59 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:00.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:59 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:00.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:59 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:14:00.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:59 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:14:00.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:59 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:00.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:59 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:00.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:59 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:14:00.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:13:59 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:00.558 DEBUG:teuthology.orchestra.run.smithi157:osd.2> sudo journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@osd.2.service 2023-06-23T01:14:00.560 INFO:tasks.cephadm:Deploying osd.3 on smithi120 with /dev/vg_nvme/lv_3... 2023-06-23T01:14:00.561 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-06-23T01:14:00.590 INFO:journalctl@ceph.osd.2.smithi157.stdout:-- Logs begin at Fri 2023-06-23 00:49:55 UTC. -- 2023-06-23T01:14:01.195 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:01 smithi157 ceph-mon[116945]: from='osd.2 [v2:172.21.15.157:6800/3996265223,v1:172.21.15.157:6801/3996265223]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-06-23T01:14:01.195 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:01 smithi157 ceph-mon[116945]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-06-23T01:14:01.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:01 smithi120 ceph-mon[79813]: from='osd.2 [v2:172.21.15.157:6800/3996265223,v1:172.21.15.157:6801/3996265223]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-06-23T01:14:01.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:01 smithi120 ceph-mon[79813]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-06-23T01:14:01.324 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T01:14:01.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:01 smithi134 ceph-mon[116146]: from='osd.2 [v2:172.21.15.157:6800/3996265223,v1:172.21.15.157:6801/3996265223]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-06-23T01:14:01.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:01 smithi134 ceph-mon[116146]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-06-23T01:14:02.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:02 smithi120 ceph-mon[79813]: pgmap v76: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-23T01:14:02.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:02 smithi120 ceph-mon[79813]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-06-23T01:14:02.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:02 smithi120 ceph-mon[79813]: osdmap e16: 3 total, 2 up, 3 in 2023-06-23T01:14:02.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:02 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-23T01:14:02.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:02 smithi120 ceph-mon[79813]: from='osd.2 [v2:172.21.15.157:6800/3996265223,v1:172.21.15.157:6801/3996265223]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-06-23T01:14:02.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:02 smithi120 ceph-mon[79813]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-06-23T01:14:02.376 INFO:journalctl@ceph.osd.2.smithi157.stdout:Jun 23 01:14:02 smithi157 conmon[124323]: 2023-06-23T01:14:02.019+0000 7fe647a4a700 -1 osd.2 0 waiting for initial osdmap 2023-06-23T01:14:02.377 INFO:journalctl@ceph.osd.2.smithi157.stdout:Jun 23 01:14:02 smithi157 conmon[124323]: 2023-06-23T01:14:02.023+0000 7fe640841700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T01:14:02.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:02 smithi157 ceph-mon[116945]: pgmap v76: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-23T01:14:02.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:02 smithi157 ceph-mon[116945]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-06-23T01:14:02.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:02 smithi157 ceph-mon[116945]: osdmap e16: 3 total, 2 up, 3 in 2023-06-23T01:14:02.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:02 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-23T01:14:02.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:02 smithi157 ceph-mon[116945]: from='osd.2 [v2:172.21.15.157:6800/3996265223,v1:172.21.15.157:6801/3996265223]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-06-23T01:14:02.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:02 smithi157 ceph-mon[116945]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-06-23T01:14:02.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:02 smithi134 ceph-mon[116146]: pgmap v76: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-23T01:14:02.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:02 smithi134 ceph-mon[116146]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-06-23T01:14:02.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:02 smithi134 ceph-mon[116146]: osdmap e16: 3 total, 2 up, 3 in 2023-06-23T01:14:02.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:02 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-23T01:14:02.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:02 smithi134 ceph-mon[116146]: from='osd.2 [v2:172.21.15.157:6800/3996265223,v1:172.21.15.157:6801/3996265223]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-06-23T01:14:02.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:02 smithi134 ceph-mon[116146]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-06-23T01:14:03.103 INFO:teuthology.orchestra.run.smithi120.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-06-23T01:14:03.103 INFO:teuthology.orchestra.run.smithi120.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-06-23T01:14:03.104 INFO:teuthology.orchestra.run.smithi120.stderr: stderr: 10+0 records in 2023-06-23T01:14:03.104 INFO:teuthology.orchestra.run.smithi120.stderr:10+0 records out 2023-06-23T01:14:03.104 INFO:teuthology.orchestra.run.smithi120.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0306118 s, 343 MB/s 2023-06-23T01:14:03.104 INFO:teuthology.orchestra.run.smithi120.stderr:--> Zapping successful for: 2023-06-23T01:14:03.291 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:03 smithi120 ceph-mon[79813]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2023-06-23T01:14:03.291 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:03 smithi120 ceph-mon[79813]: osdmap e17: 3 total, 2 up, 3 in 2023-06-23T01:14:03.291 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:03 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-23T01:14:03.291 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:03 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-23T01:14:03.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:03 smithi157 ceph-mon[116945]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2023-06-23T01:14:03.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:03 smithi157 ceph-mon[116945]: osdmap e17: 3 total, 2 up, 3 in 2023-06-23T01:14:03.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:03 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-23T01:14:03.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:03 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-23T01:14:03.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:03 smithi134 ceph-mon[116146]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2023-06-23T01:14:03.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:03 smithi134 ceph-mon[116146]: osdmap e17: 3 total, 2 up, 3 in 2023-06-23T01:14:03.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:03 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-23T01:14:03.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:03 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-23T01:14:03.481 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph orch daemon add osd smithi120:vg_nvme/lv_3 2023-06-23T01:14:04.248 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T01:14:04.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:04 smithi157 ceph-mon[116945]: purged_snaps scrub starts 2023-06-23T01:14:04.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:04 smithi157 ceph-mon[116945]: purged_snaps scrub ok 2023-06-23T01:14:04.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:04 smithi157 ceph-mon[116945]: pgmap v79: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-23T01:14:04.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:04 smithi157 ceph-mon[116945]: osd.2 [v2:172.21.15.157:6800/3996265223,v1:172.21.15.157:6801/3996265223] boot 2023-06-23T01:14:04.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:04 smithi157 ceph-mon[116945]: osdmap e18: 3 total, 3 up, 3 in 2023-06-23T01:14:04.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:04 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-23T01:14:04.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:04 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2023-06-23T01:14:04.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:04 smithi157 ceph-mon[116945]: Detected new or changed devices on smithi157 2023-06-23T01:14:04.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:04 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:04.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:04 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:04.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:04 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:14:04.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:04 smithi157 ceph-mon[116945]: Adjusting osd_memory_target on smithi157 to 17172M 2023-06-23T01:14:04.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:04 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:04.379 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:04 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:14:04.379 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:04 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:14:04.379 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:04 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:04.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:04 smithi134 ceph-mon[116146]: purged_snaps scrub starts 2023-06-23T01:14:04.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:04 smithi134 ceph-mon[116146]: purged_snaps scrub ok 2023-06-23T01:14:04.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:04 smithi134 ceph-mon[116146]: pgmap v79: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-23T01:14:04.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:04 smithi134 ceph-mon[116146]: osd.2 [v2:172.21.15.157:6800/3996265223,v1:172.21.15.157:6801/3996265223] boot 2023-06-23T01:14:04.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:04 smithi134 ceph-mon[116146]: osdmap e18: 3 total, 3 up, 3 in 2023-06-23T01:14:04.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:04 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-23T01:14:04.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:04 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2023-06-23T01:14:04.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:04 smithi134 ceph-mon[116146]: Detected new or changed devices on smithi157 2023-06-23T01:14:04.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:04 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:04.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:04 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:04.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:04 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:14:04.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:04 smithi134 ceph-mon[116146]: Adjusting osd_memory_target on smithi157 to 17172M 2023-06-23T01:14:04.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:04 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:04.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:04 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:14:04.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:04 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:14:04.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:04 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:04.496 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:04 smithi120 ceph-mon[79813]: purged_snaps scrub starts 2023-06-23T01:14:04.496 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:04 smithi120 ceph-mon[79813]: purged_snaps scrub ok 2023-06-23T01:14:04.496 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:04 smithi120 ceph-mon[79813]: pgmap v79: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-23T01:14:04.497 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:04 smithi120 ceph-mon[79813]: osd.2 [v2:172.21.15.157:6800/3996265223,v1:172.21.15.157:6801/3996265223] boot 2023-06-23T01:14:04.497 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:04 smithi120 ceph-mon[79813]: osdmap e18: 3 total, 3 up, 3 in 2023-06-23T01:14:04.497 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:04 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-23T01:14:04.498 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:04 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2023-06-23T01:14:04.498 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:04 smithi120 ceph-mon[79813]: Detected new or changed devices on smithi157 2023-06-23T01:14:04.498 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:04 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:04.498 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:04 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:04.499 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:04 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:14:04.499 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:04 smithi120 ceph-mon[79813]: Adjusting osd_memory_target on smithi157 to 17172M 2023-06-23T01:14:04.499 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:04 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:04.499 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:04 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:14:04.500 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:04 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:14:04.500 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:04 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:05.103 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:05 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2023-06-23T01:14:05.104 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:05 smithi120 ceph-mon[79813]: osdmap e19: 3 total, 3 up, 3 in 2023-06-23T01:14:05.104 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:05 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-06-23T01:14:05.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:05 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2023-06-23T01:14:05.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:05 smithi157 ceph-mon[116945]: osdmap e19: 3 total, 3 up, 3 in 2023-06-23T01:14:05.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:05 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-06-23T01:14:05.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:05 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2023-06-23T01:14:05.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:05 smithi134 ceph-mon[116146]: osdmap e19: 3 total, 3 up, 3 in 2023-06-23T01:14:05.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:05 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-06-23T01:14:05.698 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.694+0000 7f2f2a223700 1 -- 172.21.15.120:0/4171992388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f24100070 msgr2=0x7f2f24102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:14:05.699 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.694+0000 7f2f2a223700 1 --2- 172.21.15.120:0/4171992388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f24100070 0x7f2f24102450 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2f180049c0 tx=0x7f2f18009e30 comp rx=0 tx=0).stop 2023-06-23T01:14:05.699 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.695+0000 7f2f2a223700 1 -- 172.21.15.120:0/4171992388 shutdown_connections 2023-06-23T01:14:05.699 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.695+0000 7f2f2a223700 1 --2- 172.21.15.120:0/4171992388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f24102990 0x7f2f24104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:14:05.699 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.695+0000 7f2f2a223700 1 --2- 172.21.15.120:0/4171992388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f240fd750 0x7f2f240ffb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:14:05.699 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.695+0000 7f2f2a223700 1 --2- 172.21.15.120:0/4171992388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f24100070 0x7f2f24102450 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:14:05.700 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.695+0000 7f2f2a223700 1 -- 172.21.15.120:0/4171992388 >> 172.21.15.120:0/4171992388 conn(0x7f2f240f7270 msgr2=0x7f2f240f96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:14:05.700 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.695+0000 7f2f2a223700 1 -- 172.21.15.120:0/4171992388 shutdown_connections 2023-06-23T01:14:05.700 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.695+0000 7f2f2a223700 1 -- 172.21.15.120:0/4171992388 wait complete. 2023-06-23T01:14:05.700 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.695+0000 7f2f2a223700 1 Processor -- start 2023-06-23T01:14:05.700 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.696+0000 7f2f2a223700 1 -- start start 2023-06-23T01:14:05.702 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.696+0000 7f2f2a223700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f240fd750 0x7f2f2410d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:14:05.702 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.696+0000 7f2f2a223700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f24100070 0x7f2f2410dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:14:05.702 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.696+0000 7f2f2a223700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f24102990 0x7f2f24114e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:14:05.702 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.696+0000 7f2f2a223700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f24115420 con 0x7f2f240fd750 2023-06-23T01:14:05.702 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.696+0000 7f2f2a223700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f24115590 con 0x7f2f24102990 2023-06-23T01:14:05.702 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.696+0000 7f2f2a223700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f24115750 con 0x7f2f24100070 2023-06-23T01:14:05.702 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.696+0000 7f2f237fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f240fd750 0x7f2f2410d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:14:05.703 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.696+0000 7f2f237fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f240fd750 0x7f2f2410d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54868/0 (socket says 172.21.15.120:54868) 2023-06-23T01:14:05.703 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.696+0000 7f2f237fe700 1 -- 172.21.15.120:0/2664444865 learned_addr learned my addr 172.21.15.120:0/2664444865 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:14:05.703 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.696+0000 7f2f23fff700 1 --2- 172.21.15.120:0/2664444865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f24102990 0x7f2f24114e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:14:05.703 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.696+0000 7f2f22ffd700 1 --2- 172.21.15.120:0/2664444865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f24100070 0x7f2f2410dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:14:05.703 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.697+0000 7f2f237fe700 1 -- 172.21.15.120:0/2664444865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f24100070 msgr2=0x7f2f2410dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:14:05.703 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.697+0000 7f2f237fe700 1 --2- 172.21.15.120:0/2664444865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f24100070 0x7f2f2410dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:14:05.703 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.697+0000 7f2f237fe700 1 -- 172.21.15.120:0/2664444865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f24102990 msgr2=0x7f2f24114e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:14:05.704 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.697+0000 7f2f237fe700 1 --2- 172.21.15.120:0/2664444865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f24102990 0x7f2f24114e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:14:05.704 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.697+0000 7f2f237fe700 1 -- 172.21.15.120:0/2664444865 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f18005040 con 0x7f2f240fd750 2023-06-23T01:14:05.704 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.697+0000 7f2f237fe700 1 --2- 172.21.15.120:0/2664444865 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f240fd750 0x7f2f2410d840 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2f14002ad0 tx=0x7f2f140046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:14:05.704 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.697+0000 7f2f0bfff700 1 -- 172.21.15.120:0/2664444865 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f14007cc0 con 0x7f2f240fd750 2023-06-23T01:14:05.704 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.697+0000 7f2f0bfff700 1 -- 172.21.15.120:0/2664444865 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f1400f040 con 0x7f2f240fd750 2023-06-23T01:14:05.704 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.697+0000 7f2f2a223700 1 -- 172.21.15.120:0/2664444865 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f24115a30 con 0x7f2f240fd750 2023-06-23T01:14:05.704 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.697+0000 7f2f0bfff700 1 -- 172.21.15.120:0/2664444865 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1400d9a0 con 0x7f2f240fd750 2023-06-23T01:14:05.705 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.697+0000 7f2f2a223700 1 -- 172.21.15.120:0/2664444865 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f24115ff0 con 0x7f2f240fd750 2023-06-23T01:14:05.705 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.698+0000 7f2f2a223700 1 -- 172.21.15.120:0/2664444865 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f240f8da0 con 0x7f2f240fd750 2023-06-23T01:14:05.707 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.700+0000 7f2f0bfff700 1 -- 172.21.15.120:0/2664444865 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 15) v1 ==== 129146+0+0 (secure 0 0 0) 0x7f2f14009090 con 0x7f2f240fd750 2023-06-23T01:14:05.707 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.702+0000 7f2f0bfff700 1 --2- 172.21.15.120:0/2664444865 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f2f0c09b670 0x7f2f0c09db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:14:05.707 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.703+0000 7f2f0bfff700 1 -- 172.21.15.120:0/2664444865 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3233+0+0 (secure 0 0 0) 0x7f2f140c3cc0 con 0x7f2f240fd750 2023-06-23T01:14:05.707 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.703+0000 7f2f22ffd700 1 --2- 172.21.15.120:0/2664444865 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f2f0c09b670 0x7f2f0c09db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:14:05.708 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.703+0000 7f2f22ffd700 1 --2- 172.21.15.120:0/2664444865 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f2f0c09b670 0x7f2f0c09db20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800b3b0 tx=0x7f2f18009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:14:05.708 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.705+0000 7f2f0bfff700 1 -- 172.21.15.120:0/2664444865 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f2f14090930 con 0x7f2f240fd750 2023-06-23T01:14:05.926 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:05.923+0000 7f2f2a223700 1 -- 172.21.15.120:0/2664444865 --> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f2405de70 con 0x7f2f0c09b670 2023-06-23T01:14:06.335 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:06 smithi120 ceph-mon[79813]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-06-23T01:14:06.335 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:06 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-06-23T01:14:06.335 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:06 smithi120 ceph-mon[79813]: osdmap e20: 3 total, 3 up, 3 in 2023-06-23T01:14:06.335 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:06 smithi120 ceph-mon[79813]: from='client.14253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:14:06.335 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:06 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T01:14:06.336 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:06 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T01:14:06.336 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:06 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:14:06.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:06 smithi157 ceph-mon[116945]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-06-23T01:14:06.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:06 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-06-23T01:14:06.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:06 smithi157 ceph-mon[116945]: osdmap e20: 3 total, 3 up, 3 in 2023-06-23T01:14:06.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:06 smithi157 ceph-mon[116945]: from='client.14253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:14:06.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:06 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T01:14:06.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:06 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T01:14:06.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:06 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:14:06.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:06 smithi134 ceph-mon[116146]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-06-23T01:14:06.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:06 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-06-23T01:14:06.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:06 smithi134 ceph-mon[116146]: osdmap e20: 3 total, 3 up, 3 in 2023-06-23T01:14:06.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:06 smithi134 ceph-mon[116146]: from='client.14253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:14:06.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:06 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T01:14:06.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:06 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T01:14:06.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:06 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:14:06.760 INFO:journalctl@ceph.osd.0.smithi120.stdout:Jun 23 01:14:06 smithi120 sudo[99883]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-06-23T01:14:06.760 INFO:journalctl@ceph.osd.0.smithi120.stdout:Jun 23 01:14:06 smithi120 sudo[99883]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-23T01:14:06.761 INFO:journalctl@ceph.osd.0.smithi120.stdout:Jun 23 01:14:06 smithi120 sudo[99883]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-23T01:14:06.761 INFO:journalctl@ceph.osd.0.smithi120.stdout:Jun 23 01:14:06 smithi120 sudo[99883]: pam_unix(sudo:session): session closed for user root 2023-06-23T01:14:07.201 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:07 smithi120 ceph-mon[79813]: osdmap e21: 3 total, 3 up, 3 in 2023-06-23T01:14:07.202 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:07 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:07.202 INFO:journalctl@ceph.osd.0.smithi120.stdout:Jun 23 01:14:06 smithi120 sudo[100021]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-06-23T01:14:07.202 INFO:journalctl@ceph.osd.0.smithi120.stdout:Jun 23 01:14:06 smithi120 sudo[100021]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-23T01:14:07.202 INFO:journalctl@ceph.osd.0.smithi120.stdout:Jun 23 01:14:06 smithi120 sudo[100021]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-23T01:14:07.203 INFO:journalctl@ceph.osd.0.smithi120.stdout:Jun 23 01:14:06 smithi120 sudo[100021]: pam_unix(sudo:session): session closed for user root 2023-06-23T01:14:07.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:07 smithi157 ceph-mon[116945]: osdmap e21: 3 total, 3 up, 3 in 2023-06-23T01:14:07.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:07 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:07.421 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:07 smithi134 ceph-mon[116146]: osdmap e21: 3 total, 3 up, 3 in 2023-06-23T01:14:07.421 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:07 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:07.421 INFO:journalctl@ceph.osd.1.smithi134.stdout:Jun 23 01:14:07 smithi134 sudo[125344]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-06-23T01:14:07.422 INFO:journalctl@ceph.osd.1.smithi134.stdout:Jun 23 01:14:07 smithi134 sudo[125344]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-23T01:14:07.422 INFO:journalctl@ceph.osd.1.smithi134.stdout:Jun 23 01:14:07 smithi134 sudo[125344]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-23T01:14:07.422 INFO:journalctl@ceph.osd.1.smithi134.stdout:Jun 23 01:14:07 smithi134 sudo[125344]: pam_unix(sudo:session): session closed for user root 2023-06-23T01:14:07.693 INFO:journalctl@ceph.osd.1.smithi134.stdout:Jun 23 01:14:07 smithi134 sudo[125347]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-06-23T01:14:07.694 INFO:journalctl@ceph.osd.1.smithi134.stdout:Jun 23 01:14:07 smithi134 sudo[125347]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-23T01:14:07.694 INFO:journalctl@ceph.osd.1.smithi134.stdout:Jun 23 01:14:07 smithi134 sudo[125347]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-23T01:14:07.694 INFO:journalctl@ceph.osd.1.smithi134.stdout:Jun 23 01:14:07 smithi134 sudo[125347]: pam_unix(sudo:session): session closed for user root 2023-06-23T01:14:07.938 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:07.934+0000 7f2f0bfff700 1 -- 172.21.15.120:0/2664444865 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7f2f14090d00 con 0x7f2f240fd750 2023-06-23T01:14:07.944 INFO:journalctl@ceph.osd.2.smithi157.stdout:Jun 23 01:14:07 smithi157 sudo[126995]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-06-23T01:14:07.945 INFO:journalctl@ceph.osd.2.smithi157.stdout:Jun 23 01:14:07 smithi157 sudo[126995]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-23T01:14:07.945 INFO:journalctl@ceph.osd.2.smithi157.stdout:Jun 23 01:14:07 smithi157 sudo[126995]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-23T01:14:07.945 INFO:journalctl@ceph.osd.2.smithi157.stdout:Jun 23 01:14:07 smithi157 sudo[126995]: pam_unix(sudo:session): session closed for user root 2023-06-23T01:14:08.376 INFO:journalctl@ceph.osd.2.smithi157.stdout:Jun 23 01:14:07 smithi157 sudo[126998]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-06-23T01:14:08.377 INFO:journalctl@ceph.osd.2.smithi157.stdout:Jun 23 01:14:07 smithi157 sudo[126998]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-23T01:14:08.377 INFO:journalctl@ceph.osd.2.smithi157.stdout:Jun 23 01:14:07 smithi157 sudo[126998]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-23T01:14:08.377 INFO:journalctl@ceph.osd.2.smithi157.stdout:Jun 23 01:14:07 smithi157 sudo[126998]: pam_unix(sudo:session): session closed for user root 2023-06-23T01:14:08.760 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:08 smithi120 sudo[100182]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-06-23T01:14:08.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:08 smithi120 sudo[100182]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-23T01:14:08.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:08 smithi120 sudo[100182]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-23T01:14:08.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:08 smithi120 sudo[100182]: pam_unix(sudo:session): session closed for user root 2023-06-23T01:14:09.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:09 smithi157 ceph-mon[116945]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-06-23T01:14:09.292 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:09 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:09.292 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:09 smithi157 ceph-mon[116945]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-06-23T01:14:09.292 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:09 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:09.293 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:09 smithi157 sudo[127001]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-06-23T01:14:09.293 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:09 smithi157 sudo[127001]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-23T01:14:09.293 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:09 smithi157 sudo[127001]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-23T01:14:09.293 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:09 smithi157 sudo[127001]: pam_unix(sudo:session): session closed for user root 2023-06-23T01:14:09.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:09 smithi134 ceph-mon[116146]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-06-23T01:14:09.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:09 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:09.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:09 smithi134 ceph-mon[116146]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-06-23T01:14:09.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:09 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:09.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:09 smithi120 sudo[100210]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-06-23T01:14:09.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:09 smithi120 sudo[100210]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-23T01:14:09.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:09 smithi120 sudo[100210]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-23T01:14:09.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:09 smithi120 sudo[100210]: pam_unix(sudo:session): session closed for user root 2023-06-23T01:14:09.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:09 smithi120 ceph-mon[79813]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-06-23T01:14:09.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:09 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:09.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:09 smithi120 ceph-mon[79813]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-06-23T01:14:09.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:09 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:09.626 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:09 smithi157 sudo[127004]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-06-23T01:14:09.627 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:09 smithi157 sudo[127004]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-23T01:14:09.627 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:09 smithi157 sudo[127004]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-23T01:14:09.627 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:09 smithi157 sudo[127004]: pam_unix(sudo:session): session closed for user root 2023-06-23T01:14:10.024 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:09 smithi134 sudo[125350]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-06-23T01:14:10.025 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:09 smithi134 sudo[125350]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-23T01:14:10.025 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:09 smithi134 sudo[125350]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-23T01:14:10.025 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:09 smithi134 sudo[125350]: pam_unix(sudo:session): session closed for user root 2023-06-23T01:14:10.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:10 smithi157 ceph-mon[116945]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-06-23T01:14:10.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:10 smithi157 ceph-mon[116945]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-06-23T01:14:10.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:10 smithi157 ceph-mon[116945]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-06-23T01:14:10.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:10 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-23T01:14:10.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:10 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T01:14:10.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:10 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T01:14:10.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:10 smithi157 ceph-mon[116945]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-06-23T01:14:10.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:10 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-23T01:14:10.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:10 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T01:14:10.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:10 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T01:14:10.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:10 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/1852082515' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d9e9739-f043-43d6-8e46-7711dcf43e74"}]: dispatch 2023-06-23T01:14:10.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:10 smithi157 ceph-mon[116945]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-06-23T01:14:10.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:10 smithi157 ceph-mon[116945]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-06-23T01:14:10.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:10 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-23T01:14:10.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:10 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T01:14:10.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:10 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T01:14:10.379 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:10 smithi157 ceph-mon[116945]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-06-23T01:14:10.379 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:10 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/1852082515' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d9e9739-f043-43d6-8e46-7711dcf43e74"}]': finished 2023-06-23T01:14:10.379 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:10 smithi157 ceph-mon[116945]: osdmap e22: 4 total, 3 up, 4 in 2023-06-23T01:14:10.379 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:10 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-23T01:14:10.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:10 smithi134 sudo[125353]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-06-23T01:14:10.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:10 smithi134 sudo[125353]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-23T01:14:10.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:10 smithi134 sudo[125353]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-23T01:14:10.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:10 smithi134 sudo[125353]: pam_unix(sudo:session): session closed for user root 2023-06-23T01:14:10.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:10 smithi134 ceph-mon[116146]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-06-23T01:14:10.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:10 smithi134 ceph-mon[116146]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-06-23T01:14:10.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:10 smithi134 ceph-mon[116146]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-06-23T01:14:10.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:10 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-23T01:14:10.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:10 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T01:14:10.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:10 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T01:14:10.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:10 smithi134 ceph-mon[116146]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-06-23T01:14:10.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:10 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-23T01:14:10.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:10 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T01:14:10.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:10 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T01:14:10.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:10 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/1852082515' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d9e9739-f043-43d6-8e46-7711dcf43e74"}]: dispatch 2023-06-23T01:14:10.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:10 smithi134 ceph-mon[116146]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-06-23T01:14:10.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:10 smithi134 ceph-mon[116146]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-06-23T01:14:10.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:10 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-23T01:14:10.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:10 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T01:14:10.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:10 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T01:14:10.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:10 smithi134 ceph-mon[116146]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-06-23T01:14:10.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:10 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/1852082515' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d9e9739-f043-43d6-8e46-7711dcf43e74"}]': finished 2023-06-23T01:14:10.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:10 smithi134 ceph-mon[116146]: osdmap e22: 4 total, 3 up, 4 in 2023-06-23T01:14:10.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:10 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-23T01:14:10.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:10 smithi120 ceph-mon[79813]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-06-23T01:14:10.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:10 smithi120 ceph-mon[79813]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-06-23T01:14:10.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:10 smithi120 ceph-mon[79813]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-06-23T01:14:10.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:10 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-23T01:14:10.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:10 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T01:14:10.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:10 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T01:14:10.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:10 smithi120 ceph-mon[79813]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-06-23T01:14:10.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:10 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-23T01:14:10.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:10 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T01:14:10.512 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:10 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T01:14:10.512 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:10 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/1852082515' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d9e9739-f043-43d6-8e46-7711dcf43e74"}]: dispatch 2023-06-23T01:14:10.512 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:10 smithi120 ceph-mon[79813]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-06-23T01:14:10.512 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:10 smithi120 ceph-mon[79813]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-06-23T01:14:10.512 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:10 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-23T01:14:10.512 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:10 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T01:14:10.512 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:10 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T01:14:10.513 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:10 smithi120 ceph-mon[79813]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-06-23T01:14:10.513 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:10 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/1852082515' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d9e9739-f043-43d6-8e46-7711dcf43e74"}]': finished 2023-06-23T01:14:10.513 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:10 smithi120 ceph-mon[79813]: osdmap e22: 4 total, 3 up, 4 in 2023-06-23T01:14:10.513 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:10 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-23T01:14:11.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:11 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/3806145079' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T01:14:11.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:11 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/3806145079' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T01:14:11.626 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:11 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/3806145079' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T01:14:13.193 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:12 smithi134 ceph-mon[116146]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2023-06-23T01:14:13.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:12 smithi120 ceph-mon[79813]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2023-06-23T01:14:13.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:12 smithi157 ceph-mon[116945]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2023-06-23T01:14:15.193 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:14 smithi134 ceph-mon[116146]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2023-06-23T01:14:15.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:14 smithi120 ceph-mon[79813]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2023-06-23T01:14:15.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:14 smithi157 ceph-mon[116945]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2023-06-23T01:14:17.181 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:16 smithi120 ceph-mon[79813]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 53 KiB/s wr, 4 op/s 2023-06-23T01:14:17.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:16 smithi134 ceph-mon[116146]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 53 KiB/s wr, 4 op/s 2023-06-23T01:14:17.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:16 smithi157 ceph-mon[116945]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 53 KiB/s wr, 4 op/s 2023-06-23T01:14:18.193 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:17 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-06-23T01:14:18.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:17 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:14:18.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:17 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-06-23T01:14:18.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:17 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:14:18.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:17 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-06-23T01:14:18.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:17 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:14:19.193 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:18 smithi134 ceph-mon[116146]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 1.3 KiB/s rd, 38 KiB/s wr, 3 op/s 2023-06-23T01:14:19.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:18 smithi134 ceph-mon[116146]: Deploying daemon osd.3 on smithi120 2023-06-23T01:14:19.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:18 smithi120 ceph-mon[79813]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 1.3 KiB/s rd, 38 KiB/s wr, 3 op/s 2023-06-23T01:14:19.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:18 smithi120 ceph-mon[79813]: Deploying daemon osd.3 on smithi120 2023-06-23T01:14:19.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:18 smithi157 ceph-mon[116945]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 1.3 KiB/s rd, 38 KiB/s wr, 3 op/s 2023-06-23T01:14:19.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:18 smithi157 ceph-mon[116945]: Deploying daemon osd.3 on smithi120 2023-06-23T01:14:21.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:20 smithi134 ceph-mon[116146]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 1.0 KiB/s rd, 29 KiB/s wr, 2 op/s 2023-06-23T01:14:21.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:20 smithi120 ceph-mon[79813]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 1.0 KiB/s rd, 29 KiB/s wr, 2 op/s 2023-06-23T01:14:21.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:20 smithi157 ceph-mon[116945]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 1.0 KiB/s rd, 29 KiB/s wr, 2 op/s 2023-06-23T01:14:23.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:22 smithi134 ceph-mon[116146]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 840 B/s rd, 24 KiB/s wr, 2 op/s 2023-06-23T01:14:23.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:22 smithi120 ceph-mon[79813]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 840 B/s rd, 24 KiB/s wr, 2 op/s 2023-06-23T01:14:23.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:22 smithi157 ceph-mon[116945]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 840 B/s rd, 24 KiB/s wr, 2 op/s 2023-06-23T01:14:24.791 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:24 smithi120 ceph-mon[79813]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-06-23T01:14:24.791 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:24 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:24.791 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:24 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:24.792 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:24 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:14:24.876 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:24 smithi157 ceph-mon[116945]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-06-23T01:14:24.876 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:24 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:24.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:24 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:24.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:24 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:14:24.943 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:24 smithi134 ceph-mon[116146]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-06-23T01:14:24.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:24 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:24.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:24 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:24.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:24 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:14:26.760 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:26 smithi120 ceph-mon[79813]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-06-23T01:14:26.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:26 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:26.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:26 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:26.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:26 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:14:26.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:26 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:14:26.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:26 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:26.876 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:26 smithi157 ceph-mon[116945]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-06-23T01:14:26.876 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:26 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:26.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:26 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:26.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:26 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:14:26.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:26 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:14:26.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:26 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:26.892 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:26.887+0000 7f2f0bfff700 1 -- 172.21.15.120:0/2664444865 <== mgr.14152 v2:172.21.15.120:6800/1770912857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2f2405de70 con 0x7f2f0c09b670 2023-06-23T01:14:26.892 INFO:teuthology.orchestra.run.smithi120.stdout:Created osd(s) 3 on host 'smithi120' 2023-06-23T01:14:26.894 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:26.890+0000 7f2f2a223700 1 -- 172.21.15.120:0/2664444865 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f2f0c09b670 msgr2=0x7f2f0c09db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:14:26.894 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:26.890+0000 7f2f2a223700 1 --2- 172.21.15.120:0/2664444865 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f2f0c09b670 0x7f2f0c09db20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800b3b0 tx=0x7f2f18009e30 comp rx=0 tx=0).stop 2023-06-23T01:14:26.894 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:26.890+0000 7f2f2a223700 1 -- 172.21.15.120:0/2664444865 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f240fd750 msgr2=0x7f2f2410d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:14:26.894 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:26.890+0000 7f2f2a223700 1 --2- 172.21.15.120:0/2664444865 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f240fd750 0x7f2f2410d840 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2f14002ad0 tx=0x7f2f140046b0 comp rx=0 tx=0).stop 2023-06-23T01:14:26.894 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:26.890+0000 7f2f2a223700 1 -- 172.21.15.120:0/2664444865 shutdown_connections 2023-06-23T01:14:26.894 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:26.890+0000 7f2f2a223700 1 --2- 172.21.15.120:0/2664444865 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f2f0c09b670 0x7f2f0c09db20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:14:26.895 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:26.890+0000 7f2f2a223700 1 --2- 172.21.15.120:0/2664444865 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f240fd750 0x7f2f2410d840 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:14:26.895 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:26.890+0000 7f2f2a223700 1 --2- 172.21.15.120:0/2664444865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f24102990 0x7f2f24114e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:14:26.895 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:26.890+0000 7f2f2a223700 1 --2- 172.21.15.120:0/2664444865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f24100070 0x7f2f2410dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:14:26.895 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:26.890+0000 7f2f2a223700 1 -- 172.21.15.120:0/2664444865 >> 172.21.15.120:0/2664444865 conn(0x7f2f240f7270 msgr2=0x7f2f240fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:14:26.895 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:26.890+0000 7f2f2a223700 1 -- 172.21.15.120:0/2664444865 shutdown_connections 2023-06-23T01:14:26.896 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:14:26.890+0000 7f2f2a223700 1 -- 172.21.15.120:0/2664444865 wait complete. 2023-06-23T01:14:26.943 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:26 smithi134 ceph-mon[116146]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-06-23T01:14:26.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:26 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:26.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:26 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:26.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:26 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:14:26.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:26 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:14:26.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:26 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:27.234 DEBUG:teuthology.orchestra.run.smithi120:osd.3> sudo journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@osd.3.service 2023-06-23T01:14:27.239 INFO:tasks.cephadm:Deploying osd.4 on smithi134 with /dev/vg_nvme/lv_3... 2023-06-23T01:14:27.239 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-06-23T01:14:27.303 INFO:journalctl@ceph.osd.3.smithi120.stdout:-- Logs begin at Fri 2023-06-23 00:53:53 UTC. -- 2023-06-23T01:14:27.760 INFO:journalctl@ceph.osd.3.smithi120.stdout:Jun 23 01:14:27 smithi120 conmon[103072]: 2023-06-23T01:14:27.498+0000 7fd253d2f540 -1 osd.3 0 log_to_monitors true 2023-06-23T01:14:27.826 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.b/config 2023-06-23T01:14:28.184 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:27 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:28.185 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:27 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:28.185 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:27 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:14:28.185 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:27 smithi120 ceph-mon[79813]: from='osd.3 [v2:172.21.15.120:6810/1716638437,v1:172.21.15.120:6811/1716638437]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-06-23T01:14:28.186 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:27 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:28.186 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:27 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:28.186 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:27 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:14:28.186 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:27 smithi134 ceph-mon[116146]: from='osd.3 [v2:172.21.15.120:6810/1716638437,v1:172.21.15.120:6811/1716638437]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-06-23T01:14:28.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:27 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:28.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:27 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:28.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:27 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:14:28.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:27 smithi157 ceph-mon[116945]: from='osd.3 [v2:172.21.15.120:6810/1716638437,v1:172.21.15.120:6811/1716638437]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-06-23T01:14:29.193 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:28 smithi134 ceph-mon[116146]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2023-06-23T01:14:29.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:28 smithi134 ceph-mon[116146]: from='osd.3 [v2:172.21.15.120:6810/1716638437,v1:172.21.15.120:6811/1716638437]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-06-23T01:14:29.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:28 smithi134 ceph-mon[116146]: osdmap e23: 4 total, 3 up, 4 in 2023-06-23T01:14:29.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-23T01:14:29.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:28 smithi134 ceph-mon[116146]: from='osd.3 [v2:172.21.15.120:6810/1716638437,v1:172.21.15.120:6811/1716638437]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2023-06-23T01:14:29.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:28 smithi120 ceph-mon[79813]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2023-06-23T01:14:29.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:28 smithi120 ceph-mon[79813]: from='osd.3 [v2:172.21.15.120:6810/1716638437,v1:172.21.15.120:6811/1716638437]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-06-23T01:14:29.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:28 smithi120 ceph-mon[79813]: osdmap e23: 4 total, 3 up, 4 in 2023-06-23T01:14:29.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:28 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-23T01:14:29.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:28 smithi120 ceph-mon[79813]: from='osd.3 [v2:172.21.15.120:6810/1716638437,v1:172.21.15.120:6811/1716638437]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2023-06-23T01:14:29.262 INFO:journalctl@ceph.osd.3.smithi120.stdout:Jun 23 01:14:28 smithi120 conmon[103072]: 2023-06-23T01:14:28.943+0000 7fd24a514700 -1 osd.3 0 waiting for initial osdmap 2023-06-23T01:14:29.262 INFO:journalctl@ceph.osd.3.smithi120.stdout:Jun 23 01:14:28 smithi120 conmon[103072]: 2023-06-23T01:14:28.951+0000 7fd24530f700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T01:14:29.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:28 smithi157 ceph-mon[116945]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2023-06-23T01:14:29.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:28 smithi157 ceph-mon[116945]: from='osd.3 [v2:172.21.15.120:6810/1716638437,v1:172.21.15.120:6811/1716638437]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-06-23T01:14:29.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:28 smithi157 ceph-mon[116945]: osdmap e23: 4 total, 3 up, 4 in 2023-06-23T01:14:29.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:28 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-23T01:14:29.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:28 smithi157 ceph-mon[116945]: from='osd.3 [v2:172.21.15.120:6810/1716638437,v1:172.21.15.120:6811/1716638437]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2023-06-23T01:14:29.514 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-06-23T01:14:29.514 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-06-23T01:14:29.514 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2023-06-23T01:14:29.514 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2023-06-23T01:14:29.515 INFO:teuthology.orchestra.run.smithi134.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0126458 s, 829 MB/s 2023-06-23T01:14:29.515 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 2023-06-23T01:14:29.515 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2023-06-23T01:14:30.254 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:29 smithi134 ceph-mon[116146]: from='osd.3 [v2:172.21.15.120:6810/1716638437,v1:172.21.15.120:6811/1716638437]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2023-06-23T01:14:30.254 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:29 smithi134 ceph-mon[116146]: osdmap e24: 4 total, 3 up, 4 in 2023-06-23T01:14:30.255 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:29 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-23T01:14:30.255 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:29 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-23T01:14:30.256 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph orch daemon add osd smithi134:vg_nvme/lv_3 2023-06-23T01:14:30.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:29 smithi120 ceph-mon[79813]: from='osd.3 [v2:172.21.15.120:6810/1716638437,v1:172.21.15.120:6811/1716638437]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2023-06-23T01:14:30.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:29 smithi120 ceph-mon[79813]: osdmap e24: 4 total, 3 up, 4 in 2023-06-23T01:14:30.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:29 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-23T01:14:30.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:29 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-23T01:14:30.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:29 smithi157 ceph-mon[116945]: from='osd.3 [v2:172.21.15.120:6810/1716638437,v1:172.21.15.120:6811/1716638437]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2023-06-23T01:14:30.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:29 smithi157 ceph-mon[116945]: osdmap e24: 4 total, 3 up, 4 in 2023-06-23T01:14:30.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:29 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-23T01:14:30.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:29 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-23T01:14:30.832 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.b/config 2023-06-23T01:14:31.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:30 smithi120 ceph-mon[79813]: purged_snaps scrub starts 2023-06-23T01:14:31.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:30 smithi120 ceph-mon[79813]: purged_snaps scrub ok 2023-06-23T01:14:31.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:30 smithi120 ceph-mon[79813]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2023-06-23T01:14:31.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:30 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-23T01:14:31.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:30 smithi120 ceph-mon[79813]: osd.3 [v2:172.21.15.120:6810/1716638437,v1:172.21.15.120:6811/1716638437] boot 2023-06-23T01:14:31.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:30 smithi120 ceph-mon[79813]: osdmap e25: 4 total, 4 up, 4 in 2023-06-23T01:14:31.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:30 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-23T01:14:31.263 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:30 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:31.263 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:30 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:31.263 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:30 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:14:31.263 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:30 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:14:31.264 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:30 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:31.264 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:30 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:14:31.264 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:30 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:14:31.265 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:30 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:31.356 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:30 smithi134 ceph-mon[116146]: purged_snaps scrub starts 2023-06-23T01:14:31.356 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:30 smithi134 ceph-mon[116146]: purged_snaps scrub ok 2023-06-23T01:14:31.357 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:30 smithi134 ceph-mon[116146]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2023-06-23T01:14:31.357 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:30 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-23T01:14:31.357 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:30 smithi134 ceph-mon[116146]: osd.3 [v2:172.21.15.120:6810/1716638437,v1:172.21.15.120:6811/1716638437] boot 2023-06-23T01:14:31.357 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:30 smithi134 ceph-mon[116146]: osdmap e25: 4 total, 4 up, 4 in 2023-06-23T01:14:31.357 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:30 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-23T01:14:31.357 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:30 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:31.357 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:30 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:31.358 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:30 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:14:31.358 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:30 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:14:31.358 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:30 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:31.358 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:30 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:14:31.358 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:30 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:14:31.358 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:30 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:31.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:30 smithi157 ceph-mon[116945]: purged_snaps scrub starts 2023-06-23T01:14:31.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:30 smithi157 ceph-mon[116945]: purged_snaps scrub ok 2023-06-23T01:14:31.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:30 smithi157 ceph-mon[116945]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2023-06-23T01:14:31.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:30 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-23T01:14:31.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:30 smithi157 ceph-mon[116945]: osd.3 [v2:172.21.15.120:6810/1716638437,v1:172.21.15.120:6811/1716638437] boot 2023-06-23T01:14:31.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:30 smithi157 ceph-mon[116945]: osdmap e25: 4 total, 4 up, 4 in 2023-06-23T01:14:31.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:30 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-23T01:14:31.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:30 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:31.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:30 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:31.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:30 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:14:31.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:30 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:14:31.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:30 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:31.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:30 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:14:31.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:30 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:14:31.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:30 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:32.123 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:31 smithi134 ceph-mon[116146]: Detected new or changed devices on smithi120 2023-06-23T01:14:32.123 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:31 smithi134 ceph-mon[116146]: Adjusting osd_memory_target on smithi120 to 8633M 2023-06-23T01:14:32.124 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:31 smithi134 ceph-mon[116146]: osdmap e26: 4 total, 4 up, 4 in 2023-06-23T01:14:32.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:31 smithi120 ceph-mon[79813]: Detected new or changed devices on smithi120 2023-06-23T01:14:32.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:31 smithi120 ceph-mon[79813]: Adjusting osd_memory_target on smithi120 to 8633M 2023-06-23T01:14:32.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:31 smithi120 ceph-mon[79813]: osdmap e26: 4 total, 4 up, 4 in 2023-06-23T01:14:32.293 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.288+0000 7ff96f1cc700 1 -- 172.21.15.134:0/3103084169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9680fde90 msgr2=0x7ff9680fe2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:14:32.294 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.288+0000 7ff96f1cc700 1 --2- 172.21.15.134:0/3103084169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9680fde90 0x7ff9680fe2e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff9540049c0 tx=0x7ff954009e30 comp rx=0 tx=0).stop 2023-06-23T01:14:32.294 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.288+0000 7ff96f1cc700 1 -- 172.21.15.134:0/3103084169 shutdown_connections 2023-06-23T01:14:32.294 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.288+0000 7ff96f1cc700 1 --2- 172.21.15.134:0/3103084169 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9680fcc80 0x7ff9680fd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:14:32.294 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.288+0000 7ff96f1cc700 1 --2- 172.21.15.134:0/3103084169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9680fe820 0x7ff9681050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:14:32.294 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.288+0000 7ff96f1cc700 1 --2- 172.21.15.134:0/3103084169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9680fde90 0x7ff9680fe2e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:14:32.294 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.288+0000 7ff96f1cc700 1 -- 172.21.15.134:0/3103084169 >> 172.21.15.134:0/3103084169 conn(0x7ff9680f83a0 msgr2=0x7ff9680fa810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:14:32.295 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.288+0000 7ff96f1cc700 1 -- 172.21.15.134:0/3103084169 shutdown_connections 2023-06-23T01:14:32.295 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.289+0000 7ff96f1cc700 1 -- 172.21.15.134:0/3103084169 wait complete. 2023-06-23T01:14:32.295 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.289+0000 7ff96f1cc700 1 Processor -- start 2023-06-23T01:14:32.295 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.289+0000 7ff96f1cc700 1 -- start start 2023-06-23T01:14:32.295 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.289+0000 7ff96f1cc700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9680fcc80 0x7ff96810da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:14:32.295 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.289+0000 7ff96f1cc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9680fde90 0x7ff96810df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:14:32.295 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.289+0000 7ff96f1cc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9680fe820 0x7ff968114fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:14:32.296 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.289+0000 7ff96f1cc700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9681155e0 con 0x7ff9680fcc80 2023-06-23T01:14:32.296 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.289+0000 7ff96f1cc700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff968115750 con 0x7ff9680fde90 2023-06-23T01:14:32.296 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.289+0000 7ff96f1cc700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff968115910 con 0x7ff9680fe820 2023-06-23T01:14:32.296 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.289+0000 7ff967fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9680fde90 0x7ff96810df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:14:32.296 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.289+0000 7ff967fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9680fde90 0x7ff96810df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:43176/0 (socket says 172.21.15.134:43176) 2023-06-23T01:14:32.296 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.289+0000 7ff967fff700 1 -- 172.21.15.134:0/4255178954 learned_addr learned my addr 172.21.15.134:0/4255178954 (peer_addr_for_me v2:172.21.15.134:0/0) 2023-06-23T01:14:32.296 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.290+0000 7ff96d769700 1 --2- 172.21.15.134:0/4255178954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9680fe820 0x7ff968114fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:14:32.296 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.290+0000 7ff96cf68700 1 --2- 172.21.15.134:0/4255178954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9680fcc80 0x7ff96810da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:14:32.297 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.290+0000 7ff967fff700 1 -- 172.21.15.134:0/4255178954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9680fe820 msgr2=0x7ff968114fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:14:32.297 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.290+0000 7ff967fff700 1 --2- 172.21.15.134:0/4255178954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9680fe820 0x7ff968114fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:14:32.297 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.290+0000 7ff967fff700 1 -- 172.21.15.134:0/4255178954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9680fcc80 msgr2=0x7ff96810da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:14:32.297 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.290+0000 7ff967fff700 1 --2- 172.21.15.134:0/4255178954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9680fcc80 0x7ff96810da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:14:32.297 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.290+0000 7ff967fff700 1 -- 172.21.15.134:0/4255178954 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff954005040 con 0x7ff9680fde90 2023-06-23T01:14:32.297 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.290+0000 7ff967fff700 1 --2- 172.21.15.134:0/4255178954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9680fde90 0x7ff96810df40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff95400bfa0 tx=0x7ff954009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:14:32.297 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.290+0000 7ff9657fa700 1 -- 172.21.15.134:0/4255178954 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95400e040 con 0x7ff9680fde90 2023-06-23T01:14:32.298 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.290+0000 7ff96f1cc700 1 -- 172.21.15.134:0/4255178954 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff968115b90 con 0x7ff9680fde90 2023-06-23T01:14:32.298 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.290+0000 7ff9657fa700 1 -- 172.21.15.134:0/4255178954 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff954009770 con 0x7ff9680fde90 2023-06-23T01:14:32.298 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.290+0000 7ff9657fa700 1 -- 172.21.15.134:0/4255178954 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff954013e70 con 0x7ff9680fde90 2023-06-23T01:14:32.298 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.290+0000 7ff96f1cc700 1 -- 172.21.15.134:0/4255178954 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff968115ff0 con 0x7ff9680fde90 2023-06-23T01:14:32.299 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.291+0000 7ff96f1cc700 1 -- 172.21.15.134:0/4255178954 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff968103c10 con 0x7ff9680fde90 2023-06-23T01:14:32.299 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.292+0000 7ff9657fa700 1 -- 172.21.15.134:0/4255178954 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7ff9540092a0 con 0x7ff9680fde90 2023-06-23T01:14:32.299 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.293+0000 7ff9657fa700 1 --2- 172.21.15.134:0/4255178954 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7ff95809b680 0x7ff95809db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:14:32.299 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.293+0000 7ff9657fa700 1 -- 172.21.15.134:0/4255178954 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3694+0+0 (secure 0 0 0) 0x7ff9540c4e70 con 0x7ff9680fde90 2023-06-23T01:14:32.299 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.293+0000 7ff96cf68700 1 --2- 172.21.15.134:0/4255178954 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7ff95809b680 0x7ff95809db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:14:32.300 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.294+0000 7ff96cf68700 1 --2- 172.21.15.134:0/4255178954 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7ff95809b680 0x7ff95809db30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff95c006fd0 tx=0x7ff95c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:14:32.300 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.297+0000 7ff9657fa700 1 -- 172.21.15.134:0/4255178954 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7ff9540918e0 con 0x7ff9680fde90 2023-06-23T01:14:32.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:31 smithi157 ceph-mon[116945]: Detected new or changed devices on smithi120 2023-06-23T01:14:32.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:31 smithi157 ceph-mon[116945]: Adjusting osd_memory_target on smithi120 to 8633M 2023-06-23T01:14:32.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:31 smithi157 ceph-mon[116945]: osdmap e26: 4 total, 4 up, 4 in 2023-06-23T01:14:32.492 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:32.489+0000 7ff96f1cc700 1 -- 172.21.15.134:0/4255178954 --> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7ff96802b240 con 0x7ff95809b680 2023-06-23T01:14:33.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:32 smithi120 ceph-mon[79813]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-06-23T01:14:33.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:32 smithi120 ceph-mon[79813]: osdmap e27: 4 total, 4 up, 4 in 2023-06-23T01:14:33.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:32 smithi120 ceph-mon[79813]: from='client.24233 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:14:33.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:32 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T01:14:33.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:32 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T01:14:33.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:32 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:14:33.279 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:32 smithi134 ceph-mon[116146]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-06-23T01:14:33.280 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:32 smithi134 ceph-mon[116146]: osdmap e27: 4 total, 4 up, 4 in 2023-06-23T01:14:33.280 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:32 smithi134 ceph-mon[116146]: from='client.24233 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:14:33.280 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:32 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T01:14:33.280 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:32 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T01:14:33.280 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:32 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:14:33.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:32 smithi157 ceph-mon[116945]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-06-23T01:14:33.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:32 smithi157 ceph-mon[116945]: osdmap e27: 4 total, 4 up, 4 in 2023-06-23T01:14:33.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:32 smithi157 ceph-mon[116945]: from='client.24233 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:14:33.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:32 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T01:14:33.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:32 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T01:14:33.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:32 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:14:34.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:33 smithi120 ceph-mon[79813]: osdmap e28: 4 total, 4 up, 4 in 2023-06-23T01:14:34.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:33 smithi120 ceph-mon[79813]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-06-23T01:14:34.346 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:33 smithi134 ceph-mon[116146]: osdmap e28: 4 total, 4 up, 4 in 2023-06-23T01:14:34.347 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:33 smithi134 ceph-mon[116146]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-06-23T01:14:34.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:33 smithi157 ceph-mon[116945]: osdmap e28: 4 total, 4 up, 4 in 2023-06-23T01:14:34.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:33 smithi157 ceph-mon[116945]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-06-23T01:14:35.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:34 smithi120 ceph-mon[79813]: osdmap e29: 4 total, 4 up, 4 in 2023-06-23T01:14:35.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:34 smithi157 ceph-mon[116945]: osdmap e29: 4 total, 4 up, 4 in 2023-06-23T01:14:35.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:34 smithi134 ceph-mon[116146]: osdmap e29: 4 total, 4 up, 4 in 2023-06-23T01:14:36.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:36 smithi157 ceph-mon[116945]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 142 KiB/s, 0 objects/s recovering 2023-06-23T01:14:36.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:36 smithi157 ceph-mon[116945]: from='client.? 172.21.15.134:0/2196000813' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a70f86d-f903-4618-be66-9aa1eac44ccd"}]: dispatch 2023-06-23T01:14:36.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:36 smithi157 ceph-mon[116945]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a70f86d-f903-4618-be66-9aa1eac44ccd"}]: dispatch 2023-06-23T01:14:36.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:36 smithi157 ceph-mon[116945]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a70f86d-f903-4618-be66-9aa1eac44ccd"}]': finished 2023-06-23T01:14:36.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:36 smithi157 ceph-mon[116945]: osdmap e30: 5 total, 4 up, 5 in 2023-06-23T01:14:36.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:36 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-23T01:14:36.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:36 smithi157 ceph-mon[116945]: from='client.? 172.21.15.134:0/2102912694' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T01:14:36.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:36 smithi134 ceph-mon[116146]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 142 KiB/s, 0 objects/s recovering 2023-06-23T01:14:36.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:36 smithi134 ceph-mon[116146]: from='client.? 172.21.15.134:0/2196000813' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a70f86d-f903-4618-be66-9aa1eac44ccd"}]: dispatch 2023-06-23T01:14:36.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:36 smithi134 ceph-mon[116146]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a70f86d-f903-4618-be66-9aa1eac44ccd"}]: dispatch 2023-06-23T01:14:36.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:36 smithi134 ceph-mon[116146]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a70f86d-f903-4618-be66-9aa1eac44ccd"}]': finished 2023-06-23T01:14:36.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:36 smithi134 ceph-mon[116146]: osdmap e30: 5 total, 4 up, 5 in 2023-06-23T01:14:36.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:36 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-23T01:14:36.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:36 smithi134 ceph-mon[116146]: from='client.? 172.21.15.134:0/2102912694' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T01:14:36.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:36 smithi120 ceph-mon[79813]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 142 KiB/s, 0 objects/s recovering 2023-06-23T01:14:36.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:36 smithi120 ceph-mon[79813]: from='client.? 172.21.15.134:0/2196000813' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a70f86d-f903-4618-be66-9aa1eac44ccd"}]: dispatch 2023-06-23T01:14:36.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:36 smithi120 ceph-mon[79813]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a70f86d-f903-4618-be66-9aa1eac44ccd"}]: dispatch 2023-06-23T01:14:36.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:36 smithi120 ceph-mon[79813]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a70f86d-f903-4618-be66-9aa1eac44ccd"}]': finished 2023-06-23T01:14:36.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:36 smithi120 ceph-mon[79813]: osdmap e30: 5 total, 4 up, 5 in 2023-06-23T01:14:36.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:36 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-23T01:14:36.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:36 smithi120 ceph-mon[79813]: from='client.? 172.21.15.134:0/2102912694' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T01:14:39.193 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:38 smithi134 ceph-mon[116146]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 114 KiB/s, 0 objects/s recovering 2023-06-23T01:14:39.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:38 smithi120 ceph-mon[79813]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 114 KiB/s, 0 objects/s recovering 2023-06-23T01:14:39.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:38 smithi157 ceph-mon[116945]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 114 KiB/s, 0 objects/s recovering 2023-06-23T01:14:41.179 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:40 smithi134 ceph-mon[116146]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-06-23T01:14:41.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:40 smithi120 ceph-mon[79813]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-06-23T01:14:41.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:40 smithi157 ceph-mon[116945]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-06-23T01:14:43.193 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:42 smithi134 ceph-mon[116146]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-06-23T01:14:43.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:42 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-06-23T01:14:43.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:42 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:14:43.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:42 smithi120 ceph-mon[79813]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-06-23T01:14:43.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:42 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-06-23T01:14:43.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:42 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:14:43.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:42 smithi157 ceph-mon[116945]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-06-23T01:14:43.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:42 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-06-23T01:14:43.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:42 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:14:44.112 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:43 smithi134 ceph-mon[116146]: Deploying daemon osd.4 on smithi134 2023-06-23T01:14:44.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:43 smithi120 ceph-mon[79813]: Deploying daemon osd.4 on smithi134 2023-06-23T01:14:44.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:43 smithi157 ceph-mon[116945]: Deploying daemon osd.4 on smithi134 2023-06-23T01:14:45.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:44 smithi120 ceph-mon[79813]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-06-23T01:14:45.288 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:44 smithi134 ceph-mon[116146]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-06-23T01:14:45.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:44 smithi157 ceph-mon[116945]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-06-23T01:14:47.219 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:46 smithi134 ceph-mon[116146]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-06-23T01:14:47.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:46 smithi120 ceph-mon[79813]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-06-23T01:14:47.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:46 smithi157 ceph-mon[116945]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-06-23T01:14:48.604 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:48 smithi134 ceph-mon[116146]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-06-23T01:14:48.604 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:48 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:48.604 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:48 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:48.604 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:48 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:14:48.760 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:48 smithi120 ceph-mon[79813]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-06-23T01:14:48.760 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:48 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:48.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:48 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:48.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:48 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:14:48.876 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:48 smithi157 ceph-mon[116945]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-06-23T01:14:48.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:48 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:48.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:48 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:48.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:48 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:14:50.662 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:50.659+0000 7ff9657fa700 1 -- 172.21.15.134:0/4255178954 <== mgr.14152 v2:172.21.15.120:6800/1770912857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff96802b240 con 0x7ff95809b680 2023-06-23T01:14:50.662 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 4 on host 'smithi134' 2023-06-23T01:14:50.664 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:50.662+0000 7ff96f1cc700 1 -- 172.21.15.134:0/4255178954 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7ff95809b680 msgr2=0x7ff95809db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:14:50.665 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:50.662+0000 7ff96f1cc700 1 --2- 172.21.15.134:0/4255178954 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7ff95809b680 0x7ff95809db30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff95c006fd0 tx=0x7ff95c008040 comp rx=0 tx=0).stop 2023-06-23T01:14:50.665 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:50.662+0000 7ff96f1cc700 1 -- 172.21.15.134:0/4255178954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9680fde90 msgr2=0x7ff96810df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:14:50.665 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:50.662+0000 7ff96f1cc700 1 --2- 172.21.15.134:0/4255178954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9680fde90 0x7ff96810df40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff95400bfa0 tx=0x7ff954009e30 comp rx=0 tx=0).stop 2023-06-23T01:14:50.665 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:50.662+0000 7ff96f1cc700 1 -- 172.21.15.134:0/4255178954 shutdown_connections 2023-06-23T01:14:50.665 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:50.662+0000 7ff96f1cc700 1 --2- 172.21.15.134:0/4255178954 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7ff95809b680 0x7ff95809db30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:14:50.665 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:50.662+0000 7ff96f1cc700 1 --2- 172.21.15.134:0/4255178954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9680fcc80 0x7ff96810da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:14:50.666 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:50.662+0000 7ff96f1cc700 1 --2- 172.21.15.134:0/4255178954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9680fe820 0x7ff968114fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:14:50.666 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:50.662+0000 7ff96f1cc700 1 --2- 172.21.15.134:0/4255178954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9680fde90 0x7ff96810df40 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:14:50.666 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:50.662+0000 7ff96f1cc700 1 -- 172.21.15.134:0/4255178954 >> 172.21.15.134:0/4255178954 conn(0x7ff9680f83a0 msgr2=0x7ff9680fa700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:14:50.666 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:50.662+0000 7ff96f1cc700 1 -- 172.21.15.134:0/4255178954 shutdown_connections 2023-06-23T01:14:50.666 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:14:50.662+0000 7ff96f1cc700 1 -- 172.21.15.134:0/4255178954 wait complete. 2023-06-23T01:14:50.886 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:50 smithi134 ceph-mon[116146]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-06-23T01:14:50.887 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:50 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:50.887 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:50 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:50.887 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:50 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:14:50.887 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:50 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:14:50.887 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:50 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:50.887 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:50 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:50.887 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:50 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:14:50.888 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:50 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:51.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:50 smithi157 ceph-mon[116945]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-06-23T01:14:51.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:50 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:51.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:50 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:51.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:50 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:14:51.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:50 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:14:51.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:50 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:51.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:50 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:51.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:50 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:14:51.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:50 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:51.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:50 smithi120 ceph-mon[79813]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-06-23T01:14:51.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:50 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:51.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:50 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:51.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:50 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:14:51.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:50 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:14:51.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:50 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:51.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:50 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:51.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:50 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:14:51.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:50 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:51.284 DEBUG:teuthology.orchestra.run.smithi134:osd.4> sudo journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@osd.4.service 2023-06-23T01:14:51.286 INFO:tasks.cephadm:Deploying osd.5 on smithi157 with /dev/vg_nvme/lv_3... 2023-06-23T01:14:51.287 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-06-23T01:14:51.325 INFO:journalctl@ceph.osd.4.smithi134.stdout:-- Logs begin at Fri 2023-06-23 00:49:19 UTC. -- 2023-06-23T01:14:51.629 INFO:journalctl@ceph.osd.4.smithi134.stdout:Jun 23 01:14:51 smithi134 conmon[129635]: 2023-06-23T01:14:51.402+0000 7fb42cb6a540 -1 osd.4 0 log_to_monitors true 2023-06-23T01:14:51.886 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.c/config 2023-06-23T01:14:51.943 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:51 smithi134 ceph-mon[116146]: from='osd.4 [v2:172.21.15.134:6808/4172238262,v1:172.21.15.134:6809/4172238262]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-06-23T01:14:51.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:51 smithi134 ceph-mon[116146]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-06-23T01:14:52.121 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:51 smithi157 ceph-mon[116945]: from='osd.4 [v2:172.21.15.134:6808/4172238262,v1:172.21.15.134:6809/4172238262]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-06-23T01:14:52.122 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:51 smithi157 ceph-mon[116945]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-06-23T01:14:52.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:51 smithi120 ceph-mon[79813]: from='osd.4 [v2:172.21.15.134:6808/4172238262,v1:172.21.15.134:6809/4172238262]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-06-23T01:14:52.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:51 smithi120 ceph-mon[79813]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-06-23T01:14:53.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:52 smithi157 ceph-mon[116945]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-06-23T01:14:53.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:52 smithi157 ceph-mon[116945]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-06-23T01:14:53.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:52 smithi157 ceph-mon[116945]: osdmap e31: 5 total, 4 up, 5 in 2023-06-23T01:14:53.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:52 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-23T01:14:53.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:52 smithi157 ceph-mon[116945]: from='osd.4 [v2:172.21.15.134:6808/4172238262,v1:172.21.15.134:6809/4172238262]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-06-23T01:14:53.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:52 smithi157 ceph-mon[116945]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-06-23T01:14:53.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:52 smithi134 ceph-mon[116146]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-06-23T01:14:53.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:52 smithi134 ceph-mon[116146]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-06-23T01:14:53.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:52 smithi134 ceph-mon[116146]: osdmap e31: 5 total, 4 up, 5 in 2023-06-23T01:14:53.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:52 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-23T01:14:53.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:52 smithi134 ceph-mon[116146]: from='osd.4 [v2:172.21.15.134:6808/4172238262,v1:172.21.15.134:6809/4172238262]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-06-23T01:14:53.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:52 smithi134 ceph-mon[116146]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-06-23T01:14:53.195 INFO:journalctl@ceph.osd.4.smithi134.stdout:Jun 23 01:14:52 smithi134 conmon[129635]: 2023-06-23T01:14:52.868+0000 7fb42334f700 -1 osd.4 0 waiting for initial osdmap 2023-06-23T01:14:53.195 INFO:journalctl@ceph.osd.4.smithi134.stdout:Jun 23 01:14:52 smithi134 conmon[129635]: 2023-06-23T01:14:52.877+0000 7fb41e94b700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T01:14:53.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:52 smithi120 ceph-mon[79813]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-06-23T01:14:53.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:52 smithi120 ceph-mon[79813]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-06-23T01:14:53.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:52 smithi120 ceph-mon[79813]: osdmap e31: 5 total, 4 up, 5 in 2023-06-23T01:14:53.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:52 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-23T01:14:53.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:52 smithi120 ceph-mon[79813]: from='osd.4 [v2:172.21.15.134:6808/4172238262,v1:172.21.15.134:6809/4172238262]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-06-23T01:14:53.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:52 smithi120 ceph-mon[79813]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-06-23T01:14:53.538 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-06-23T01:14:53.538 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-06-23T01:14:53.538 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2023-06-23T01:14:53.538 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2023-06-23T01:14:53.538 INFO:teuthology.orchestra.run.smithi157.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00809824 s, 1.3 GB/s 2023-06-23T01:14:53.538 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2023-06-23T01:14:54.119 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph orch daemon add osd smithi157:vg_nvme/lv_3 2023-06-23T01:14:54.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:53 smithi157 ceph-mon[116945]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-06-23T01:14:54.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:53 smithi157 ceph-mon[116945]: osdmap e32: 5 total, 4 up, 5 in 2023-06-23T01:14:54.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:53 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-23T01:14:54.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:53 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-23T01:14:54.193 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:53 smithi134 ceph-mon[116146]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-06-23T01:14:54.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:53 smithi134 ceph-mon[116146]: osdmap e32: 5 total, 4 up, 5 in 2023-06-23T01:14:54.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:53 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-23T01:14:54.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:53 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-23T01:14:54.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:53 smithi120 ceph-mon[79813]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-06-23T01:14:54.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:53 smithi120 ceph-mon[79813]: osdmap e32: 5 total, 4 up, 5 in 2023-06-23T01:14:54.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:53 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-23T01:14:54.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:53 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-23T01:14:54.701 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.c/config 2023-06-23T01:14:55.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:54 smithi157 ceph-mon[116945]: purged_snaps scrub starts 2023-06-23T01:14:55.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:54 smithi157 ceph-mon[116945]: purged_snaps scrub ok 2023-06-23T01:14:55.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:54 smithi157 ceph-mon[116945]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-06-23T01:14:55.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:54 smithi157 ceph-mon[116945]: osd.4 [v2:172.21.15.134:6808/4172238262,v1:172.21.15.134:6809/4172238262] boot 2023-06-23T01:14:55.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:54 smithi157 ceph-mon[116945]: osdmap e33: 5 total, 5 up, 5 in 2023-06-23T01:14:55.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:54 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-23T01:14:55.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:54 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:55.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:54 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:55.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:54 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:14:55.129 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:54 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:14:55.129 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:54 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:55.129 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:54 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:14:55.129 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:54 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:14:55.129 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:54 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:55.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:54 smithi134 ceph-mon[116146]: purged_snaps scrub starts 2023-06-23T01:14:55.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:54 smithi134 ceph-mon[116146]: purged_snaps scrub ok 2023-06-23T01:14:55.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:54 smithi134 ceph-mon[116146]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-06-23T01:14:55.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:54 smithi134 ceph-mon[116146]: osd.4 [v2:172.21.15.134:6808/4172238262,v1:172.21.15.134:6809/4172238262] boot 2023-06-23T01:14:55.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:54 smithi134 ceph-mon[116146]: osdmap e33: 5 total, 5 up, 5 in 2023-06-23T01:14:55.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:54 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-23T01:14:55.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:54 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:55.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:54 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:55.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:54 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:14:55.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:54 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:14:55.196 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:54 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:55.196 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:54 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:14:55.196 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:54 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:14:55.196 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:54 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:55.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:54 smithi120 ceph-mon[79813]: purged_snaps scrub starts 2023-06-23T01:14:55.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:54 smithi120 ceph-mon[79813]: purged_snaps scrub ok 2023-06-23T01:14:55.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:54 smithi120 ceph-mon[79813]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-06-23T01:14:55.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:54 smithi120 ceph-mon[79813]: osd.4 [v2:172.21.15.134:6808/4172238262,v1:172.21.15.134:6809/4172238262] boot 2023-06-23T01:14:55.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:54 smithi120 ceph-mon[79813]: osdmap e33: 5 total, 5 up, 5 in 2023-06-23T01:14:55.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:54 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-23T01:14:55.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:54 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:55.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:54 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:55.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:54 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:14:55.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:54 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:14:55.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:54 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:55.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:54 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:14:55.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:54 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:14:55.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:54 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:14:56.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.050+0000 7f9d3c9c8700 1 -- 172.21.15.157:0/76958208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d380fbb20 msgr2=0x7f9d380fbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:14:56.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.050+0000 7f9d3c9c8700 1 --2- 172.21.15.157:0/76958208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d380fbb20 0x7f9d380fbf30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9d280049c0 tx=0x7f9d28009e30 comp rx=0 tx=0).stop 2023-06-23T01:14:56.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.051+0000 7f9d3c9c8700 1 -- 172.21.15.157:0/76958208 shutdown_connections 2023-06-23T01:14:56.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.051+0000 7f9d3c9c8700 1 --2- 172.21.15.157:0/76958208 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d380fd6c0 0x7f9d38103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:14:56.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.051+0000 7f9d3c9c8700 1 --2- 172.21.15.157:0/76958208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d380fbb20 0x7f9d380fbf30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:14:56.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.051+0000 7f9d3c9c8700 1 --2- 172.21.15.157:0/76958208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d380fcd30 0x7f9d380fd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:14:56.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.051+0000 7f9d3c9c8700 1 -- 172.21.15.157:0/76958208 >> 172.21.15.157:0/76958208 conn(0x7f9d380f7260 msgr2=0x7f9d380f96b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:14:56.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.051+0000 7f9d3c9c8700 1 -- 172.21.15.157:0/76958208 shutdown_connections 2023-06-23T01:14:56.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.051+0000 7f9d3c9c8700 1 -- 172.21.15.157:0/76958208 wait complete. 2023-06-23T01:14:56.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.051+0000 7f9d3c9c8700 1 Processor -- start 2023-06-23T01:14:56.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.051+0000 7f9d3c9c8700 1 -- start start 2023-06-23T01:14:56.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.051+0000 7f9d3c9c8700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d380fbb20 0x7f9d3810c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:14:56.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.051+0000 7f9d3c9c8700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d380fcd30 0x7f9d3810cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:14:56.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.051+0000 7f9d3c9c8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d380fd6c0 0x7f9d38113e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:14:56.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.051+0000 7f9d3c9c8700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d38114460 con 0x7f9d380fbb20 2023-06-23T01:14:56.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.051+0000 7f9d3c9c8700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d381145d0 con 0x7f9d380fcd30 2023-06-23T01:14:56.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.051+0000 7f9d3c9c8700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d38114790 con 0x7f9d380fd6c0 2023-06-23T01:14:56.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.051+0000 7f9d36d9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d380fd6c0 0x7f9d38113e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:14:56.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.051+0000 7f9d36d9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d380fd6c0 0x7f9d38113e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39064/0 (socket says 172.21.15.157:39064) 2023-06-23T01:14:56.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.051+0000 7f9d36d9d700 1 -- 172.21.15.157:0/86289634 learned_addr learned my addr 172.21.15.157:0/86289634 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-06-23T01:14:56.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.051+0000 7f9d35d9b700 1 --2- 172.21.15.157:0/86289634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d380fcd30 0x7f9d3810cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:14:56.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.052+0000 7f9d3659c700 1 --2- 172.21.15.157:0/86289634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d380fbb20 0x7f9d3810c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:14:56.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.052+0000 7f9d36d9d700 1 -- 172.21.15.157:0/86289634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d380fcd30 msgr2=0x7f9d3810cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:14:56.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.052+0000 7f9d36d9d700 1 --2- 172.21.15.157:0/86289634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d380fcd30 0x7f9d3810cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:14:56.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.052+0000 7f9d36d9d700 1 -- 172.21.15.157:0/86289634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d380fbb20 msgr2=0x7f9d3810c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:14:56.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.052+0000 7f9d36d9d700 1 --2- 172.21.15.157:0/86289634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d380fbb20 0x7f9d3810c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:14:56.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.052+0000 7f9d36d9d700 1 -- 172.21.15.157:0/86289634 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d28005040 con 0x7f9d380fd6c0 2023-06-23T01:14:56.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.052+0000 7f9d3659c700 1 --2- 172.21.15.157:0/86289634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d380fbb20 0x7f9d3810c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T01:14:56.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.052+0000 7f9d36d9d700 1 --2- 172.21.15.157:0/86289634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d380fd6c0 0x7f9d38113e50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9d3000bc30 tx=0x7f9d30009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:14:56.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.052+0000 7f9d26ffd700 1 -- 172.21.15.157:0/86289634 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d300092e0 con 0x7f9d380fd6c0 2023-06-23T01:14:56.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.052+0000 7f9d3c9c8700 1 -- 172.21.15.157:0/86289634 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d38114a70 con 0x7f9d380fd6c0 2023-06-23T01:14:56.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.052+0000 7f9d26ffd700 1 -- 172.21.15.157:0/86289634 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d3000cbc0 con 0x7f9d380fd6c0 2023-06-23T01:14:56.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.052+0000 7f9d26ffd700 1 -- 172.21.15.157:0/86289634 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d30015cb0 con 0x7f9d380fd6c0 2023-06-23T01:14:56.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.052+0000 7f9d3c9c8700 1 -- 172.21.15.157:0/86289634 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d38114f10 con 0x7f9d380fd6c0 2023-06-23T01:14:56.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.053+0000 7f9d3c9c8700 1 -- 172.21.15.157:0/86289634 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d38107390 con 0x7f9d380fd6c0 2023-06-23T01:14:56.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.054+0000 7f9d26ffd700 1 -- 172.21.15.157:0/86289634 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7f9d30012070 con 0x7f9d380fd6c0 2023-06-23T01:14:56.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.054+0000 7f9d26ffd700 1 --2- 172.21.15.157:0/86289634 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f9d2009b680 0x7f9d2009db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:14:56.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.055+0000 7f9d26ffd700 1 -- 172.21.15.157:0/86289634 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4097+0+0 (secure 0 0 0) 0x7f9d300c3d20 con 0x7f9d380fd6c0 2023-06-23T01:14:56.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.055+0000 7f9d3659c700 1 --2- 172.21.15.157:0/86289634 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f9d2009b680 0x7f9d2009db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:14:56.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.055+0000 7f9d3659c700 1 --2- 172.21.15.157:0/86289634 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f9d2009b680 0x7f9d2009db30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9d28005010 tx=0x7f9d28009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:14:56.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.058+0000 7f9d26ffd700 1 -- 172.21.15.157:0/86289634 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f9d300905f0 con 0x7f9d380fd6c0 2023-06-23T01:14:56.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:55 smithi157 ceph-mon[116945]: Detected new or changed devices on smithi134 2023-06-23T01:14:56.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:55 smithi157 ceph-mon[116945]: Adjusting osd_memory_target on smithi134 to 8586M 2023-06-23T01:14:56.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:55 smithi157 ceph-mon[116945]: osdmap e34: 5 total, 5 up, 5 in 2023-06-23T01:14:56.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:55 smithi134 ceph-mon[116146]: Detected new or changed devices on smithi134 2023-06-23T01:14:56.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:55 smithi134 ceph-mon[116146]: Adjusting osd_memory_target on smithi134 to 8586M 2023-06-23T01:14:56.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:55 smithi134 ceph-mon[116146]: osdmap e34: 5 total, 5 up, 5 in 2023-06-23T01:14:56.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:14:56.245+0000 7f9d3c9c8700 1 -- 172.21.15.157:0/86289634 --> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d3805de70 con 0x7f9d2009b680 2023-06-23T01:14:56.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:55 smithi120 ceph-mon[79813]: Detected new or changed devices on smithi134 2023-06-23T01:14:56.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:55 smithi120 ceph-mon[79813]: Adjusting osd_memory_target on smithi134 to 8586M 2023-06-23T01:14:56.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:55 smithi120 ceph-mon[79813]: osdmap e34: 5 total, 5 up, 5 in 2023-06-23T01:14:57.140 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:56 smithi157 ceph-mon[116945]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2023-06-23T01:14:57.140 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:56 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T01:14:57.140 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:56 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T01:14:57.141 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:56 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:14:57.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:56 smithi134 ceph-mon[116146]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2023-06-23T01:14:57.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:56 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T01:14:57.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:56 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T01:14:57.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:56 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:14:57.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:56 smithi120 ceph-mon[79813]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2023-06-23T01:14:57.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:56 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T01:14:57.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:56 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T01:14:57.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:56 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:14:58.093 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:57 smithi157 ceph-mon[116945]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:14:58.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:57 smithi134 ceph-mon[116146]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:14:58.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:57 smithi120 ceph-mon[79813]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:14:59.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:58 smithi120 ceph-mon[79813]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2023-06-23T01:14:59.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:58 smithi120 ceph-mon[79813]: from='client.? 172.21.15.157:0/390263605' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b2316488-75c3-4847-b4c4-998730f27c88"}]: dispatch 2023-06-23T01:14:59.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:58 smithi120 ceph-mon[79813]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b2316488-75c3-4847-b4c4-998730f27c88"}]: dispatch 2023-06-23T01:14:59.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:58 smithi120 ceph-mon[79813]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b2316488-75c3-4847-b4c4-998730f27c88"}]': finished 2023-06-23T01:14:59.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:58 smithi120 ceph-mon[79813]: osdmap e35: 6 total, 5 up, 6 in 2023-06-23T01:14:59.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:58 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-23T01:14:59.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:58 smithi157 ceph-mon[116945]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2023-06-23T01:14:59.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:58 smithi157 ceph-mon[116945]: from='client.? 172.21.15.157:0/390263605' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b2316488-75c3-4847-b4c4-998730f27c88"}]: dispatch 2023-06-23T01:14:59.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:58 smithi157 ceph-mon[116945]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b2316488-75c3-4847-b4c4-998730f27c88"}]: dispatch 2023-06-23T01:14:59.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:58 smithi157 ceph-mon[116945]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b2316488-75c3-4847-b4c4-998730f27c88"}]': finished 2023-06-23T01:14:59.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:58 smithi157 ceph-mon[116945]: osdmap e35: 6 total, 5 up, 6 in 2023-06-23T01:14:59.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:58 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-23T01:14:59.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:58 smithi134 ceph-mon[116146]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2023-06-23T01:14:59.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:58 smithi134 ceph-mon[116146]: from='client.? 172.21.15.157:0/390263605' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b2316488-75c3-4847-b4c4-998730f27c88"}]: dispatch 2023-06-23T01:14:59.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:58 smithi134 ceph-mon[116146]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b2316488-75c3-4847-b4c4-998730f27c88"}]: dispatch 2023-06-23T01:14:59.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:58 smithi134 ceph-mon[116146]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b2316488-75c3-4847-b4c4-998730f27c88"}]': finished 2023-06-23T01:14:59.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:58 smithi134 ceph-mon[116146]: osdmap e35: 6 total, 5 up, 6 in 2023-06-23T01:14:59.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:58 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-23T01:15:00.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:14:59 smithi120 ceph-mon[79813]: from='client.? 172.21.15.157:0/2570020930' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T01:15:00.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:14:59 smithi157 ceph-mon[116945]: from='client.? 172.21.15.157:0/2570020930' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T01:15:00.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:14:59 smithi134 ceph-mon[116146]: from='client.? 172.21.15.157:0/2570020930' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T01:15:01.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:00 smithi120 ceph-mon[79813]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2023-06-23T01:15:01.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:00 smithi157 ceph-mon[116945]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2023-06-23T01:15:01.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:00 smithi134 ceph-mon[116146]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2023-06-23T01:15:03.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:02 smithi120 ceph-mon[79813]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-06-23T01:15:03.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:02 smithi157 ceph-mon[116945]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-06-23T01:15:03.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:02 smithi134 ceph-mon[116146]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-06-23T01:15:05.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:04 smithi120 ceph-mon[79813]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-06-23T01:15:05.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:04 smithi157 ceph-mon[116945]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-06-23T01:15:05.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:04 smithi134 ceph-mon[116146]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-06-23T01:15:07.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:06 smithi120 ceph-mon[79813]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-06-23T01:15:07.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:06 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-06-23T01:15:07.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:06 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:15:07.365 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:06 smithi157 ceph-mon[116945]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-06-23T01:15:07.366 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:06 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-06-23T01:15:07.366 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:06 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:15:07.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:06 smithi134 ceph-mon[116146]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-06-23T01:15:07.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:06 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-06-23T01:15:07.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:06 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:15:08.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:07 smithi120 ceph-mon[79813]: Deploying daemon osd.5 on smithi157 2023-06-23T01:15:08.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:07 smithi157 ceph-mon[116945]: Deploying daemon osd.5 on smithi157 2023-06-23T01:15:08.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:07 smithi134 ceph-mon[116146]: Deploying daemon osd.5 on smithi157 2023-06-23T01:15:09.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:08 smithi157 ceph-mon[116945]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-06-23T01:15:09.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:08 smithi120 ceph-mon[79813]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-06-23T01:15:09.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:08 smithi134 ceph-mon[116146]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-06-23T01:15:10.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:09 smithi157 ceph-mon[116945]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-06-23T01:15:10.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:09 smithi120 ceph-mon[79813]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-06-23T01:15:10.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:09 smithi134 ceph-mon[116146]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-06-23T01:15:12.250 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:12 smithi157 ceph-mon[116945]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-06-23T01:15:12.250 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:12 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:12.250 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:12 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:12.251 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:12 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:15:12.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:12 smithi134 ceph-mon[116146]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-06-23T01:15:12.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:12 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:12.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:12 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:12.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:12 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:15:12.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:12 smithi120 ceph-mon[79813]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-06-23T01:15:12.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:12 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:12.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:12 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:12.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:12 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:15:14.520 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 5 on host 'smithi157' 2023-06-23T01:15:14.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:15:14.517+0000 7f9d26ffd700 1 -- 172.21.15.157:0/86289634 <== mgr.14152 v2:172.21.15.120:6800/1770912857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9d3805de70 con 0x7f9d2009b680 2023-06-23T01:15:14.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:15:14.519+0000 7f9d3c9c8700 1 -- 172.21.15.157:0/86289634 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f9d2009b680 msgr2=0x7f9d2009db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:15:14.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:15:14.519+0000 7f9d3c9c8700 1 --2- 172.21.15.157:0/86289634 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f9d2009b680 0x7f9d2009db30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9d28005010 tx=0x7f9d28009e30 comp rx=0 tx=0).stop 2023-06-23T01:15:14.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:15:14.519+0000 7f9d3c9c8700 1 -- 172.21.15.157:0/86289634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d380fd6c0 msgr2=0x7f9d38113e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:15:14.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:15:14.519+0000 7f9d3c9c8700 1 --2- 172.21.15.157:0/86289634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d380fd6c0 0x7f9d38113e50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9d3000bc30 tx=0x7f9d30009840 comp rx=0 tx=0).stop 2023-06-23T01:15:14.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:15:14.519+0000 7f9d3c9c8700 1 -- 172.21.15.157:0/86289634 shutdown_connections 2023-06-23T01:15:14.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:15:14.519+0000 7f9d3c9c8700 1 --2- 172.21.15.157:0/86289634 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f9d2009b680 0x7f9d2009db30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:15:14.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:15:14.519+0000 7f9d3c9c8700 1 --2- 172.21.15.157:0/86289634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d380fbb20 0x7f9d3810c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:15:14.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:15:14.519+0000 7f9d3c9c8700 1 --2- 172.21.15.157:0/86289634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d380fd6c0 0x7f9d38113e50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:15:14.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:15:14.519+0000 7f9d3c9c8700 1 --2- 172.21.15.157:0/86289634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d380fcd30 0x7f9d3810cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:15:14.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:15:14.519+0000 7f9d3c9c8700 1 -- 172.21.15.157:0/86289634 >> 172.21.15.157:0/86289634 conn(0x7f9d380f7260 msgr2=0x7f9d380f9520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:15:14.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:15:14.520+0000 7f9d3c9c8700 1 -- 172.21.15.157:0/86289634 shutdown_connections 2023-06-23T01:15:14.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:15:14.520+0000 7f9d3c9c8700 1 -- 172.21.15.157:0/86289634 wait complete. 2023-06-23T01:15:14.716 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:14 smithi157 ceph-mon[116945]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-06-23T01:15:14.717 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:14 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:14.717 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:14 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:14.717 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:14 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:15:14.717 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:14 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:15:14.717 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:14 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:14.717 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:14 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:14.718 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:14 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:14.718 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:14 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:15:14.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:14 smithi134 ceph-mon[116146]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-06-23T01:15:14.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:14 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:14.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:14 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:14.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:14 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:15:14.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:14 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:15:14.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:14 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:14.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:14 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:14.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:14 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:14.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:14 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:15:15.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:14 smithi120 ceph-mon[79813]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-06-23T01:15:15.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:14 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:15.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:14 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:15.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:14 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:15:15.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:14 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:15:15.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:14 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:15.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:14 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:15.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:14 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:15.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:14 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:15:15.151 DEBUG:teuthology.orchestra.run.smithi157:osd.5> sudo journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@osd.5.service 2023-06-23T01:15:15.193 INFO:tasks.cephadm:Deploying osd.6 on smithi120 with /dev/vg_nvme/lv_2... 2023-06-23T01:15:15.193 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-06-23T01:15:15.221 INFO:journalctl@ceph.osd.5.smithi157.stdout:-- Logs begin at Fri 2023-06-23 00:49:55 UTC. -- 2023-06-23T01:15:15.849 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:15 smithi157 ceph-mon[116945]: from='osd.5 [v2:172.21.15.157:6808/3374822045,v1:172.21.15.157:6809/3374822045]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-06-23T01:15:15.850 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:15 smithi157 ceph-mon[116945]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-06-23T01:15:15.943 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:15 smithi134 ceph-mon[116146]: from='osd.5 [v2:172.21.15.157:6808/3374822045,v1:172.21.15.157:6809/3374822045]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-06-23T01:15:15.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:15 smithi134 ceph-mon[116146]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-06-23T01:15:15.971 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T01:15:16.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:15 smithi120 ceph-mon[79813]: from='osd.5 [v2:172.21.15.157:6808/3374822045,v1:172.21.15.157:6809/3374822045]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-06-23T01:15:16.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:15 smithi120 ceph-mon[79813]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-06-23T01:15:16.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:16 smithi157 ceph-mon[116945]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-06-23T01:15:16.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:16 smithi157 ceph-mon[116945]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-06-23T01:15:16.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:16 smithi157 ceph-mon[116945]: osdmap e36: 6 total, 5 up, 6 in 2023-06-23T01:15:16.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:16 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-23T01:15:16.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:16 smithi157 ceph-mon[116945]: from='osd.5 [v2:172.21.15.157:6808/3374822045,v1:172.21.15.157:6809/3374822045]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-06-23T01:15:16.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:16 smithi157 ceph-mon[116945]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-06-23T01:15:16.878 INFO:journalctl@ceph.osd.5.smithi157.stdout:Jun 23 01:15:16 smithi157 conmon[131277]: 2023-06-23T01:15:16.547+0000 7fe05611e700 -1 osd.5 0 waiting for initial osdmap 2023-06-23T01:15:16.878 INFO:journalctl@ceph.osd.5.smithi157.stdout:Jun 23 01:15:16 smithi157 conmon[131277]: 2023-06-23T01:15:16.555+0000 7fe050f19700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T01:15:16.943 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:16 smithi134 ceph-mon[116146]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-06-23T01:15:16.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:16 smithi134 ceph-mon[116146]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-06-23T01:15:16.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:16 smithi134 ceph-mon[116146]: osdmap e36: 6 total, 5 up, 6 in 2023-06-23T01:15:16.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:16 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-23T01:15:16.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:16 smithi134 ceph-mon[116146]: from='osd.5 [v2:172.21.15.157:6808/3374822045,v1:172.21.15.157:6809/3374822045]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-06-23T01:15:16.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:16 smithi134 ceph-mon[116146]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-06-23T01:15:17.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:16 smithi120 ceph-mon[79813]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-06-23T01:15:17.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:16 smithi120 ceph-mon[79813]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-06-23T01:15:17.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:16 smithi120 ceph-mon[79813]: osdmap e36: 6 total, 5 up, 6 in 2023-06-23T01:15:17.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:16 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-23T01:15:17.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:16 smithi120 ceph-mon[79813]: from='osd.5 [v2:172.21.15.157:6808/3374822045,v1:172.21.15.157:6809/3374822045]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-06-23T01:15:17.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:16 smithi120 ceph-mon[79813]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-06-23T01:15:17.802 INFO:teuthology.orchestra.run.smithi120.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-06-23T01:15:17.802 INFO:teuthology.orchestra.run.smithi120.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-06-23T01:15:17.802 INFO:teuthology.orchestra.run.smithi120.stderr: stderr: 10+0 records in 2023-06-23T01:15:17.802 INFO:teuthology.orchestra.run.smithi120.stderr:10+0 records out 2023-06-23T01:15:17.803 INFO:teuthology.orchestra.run.smithi120.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0303465 s, 346 MB/s 2023-06-23T01:15:17.803 INFO:teuthology.orchestra.run.smithi120.stderr:--> Zapping successful for: 2023-06-23T01:15:17.839 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:17 smithi120 ceph-mon[79813]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2023-06-23T01:15:17.839 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:17 smithi120 ceph-mon[79813]: osdmap e37: 6 total, 5 up, 6 in 2023-06-23T01:15:17.839 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:17 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-23T01:15:17.839 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:17 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-23T01:15:17.839 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:17 smithi120 ceph-mon[79813]: osd.5 [v2:172.21.15.157:6808/3374822045,v1:172.21.15.157:6809/3374822045] boot 2023-06-23T01:15:17.840 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:17 smithi120 ceph-mon[79813]: osdmap e38: 6 total, 6 up, 6 in 2023-06-23T01:15:17.840 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:17 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-23T01:15:17.861 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:17 smithi157 ceph-mon[116945]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2023-06-23T01:15:17.861 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:17 smithi157 ceph-mon[116945]: osdmap e37: 6 total, 5 up, 6 in 2023-06-23T01:15:17.862 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:17 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-23T01:15:17.862 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:17 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-23T01:15:17.862 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:17 smithi157 ceph-mon[116945]: osd.5 [v2:172.21.15.157:6808/3374822045,v1:172.21.15.157:6809/3374822045] boot 2023-06-23T01:15:17.862 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:17 smithi157 ceph-mon[116945]: osdmap e38: 6 total, 6 up, 6 in 2023-06-23T01:15:17.862 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:17 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-23T01:15:17.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:17 smithi134 ceph-mon[116146]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2023-06-23T01:15:17.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:17 smithi134 ceph-mon[116146]: osdmap e37: 6 total, 5 up, 6 in 2023-06-23T01:15:17.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:17 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-23T01:15:17.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:17 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-23T01:15:17.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:17 smithi134 ceph-mon[116146]: osd.5 [v2:172.21.15.157:6808/3374822045,v1:172.21.15.157:6809/3374822045] boot 2023-06-23T01:15:17.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:17 smithi134 ceph-mon[116146]: osdmap e38: 6 total, 6 up, 6 in 2023-06-23T01:15:17.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:17 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-23T01:15:18.232 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph orch daemon add osd smithi120:vg_nvme/lv_2 2023-06-23T01:15:18.987 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T01:15:19.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:18 smithi157 ceph-mon[116945]: purged_snaps scrub starts 2023-06-23T01:15:19.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:18 smithi157 ceph-mon[116945]: purged_snaps scrub ok 2023-06-23T01:15:19.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:18 smithi157 ceph-mon[116945]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-06-23T01:15:19.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:18 smithi157 ceph-mon[116945]: Detected new or changed devices on smithi157 2023-06-23T01:15:19.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:18 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:19.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:18 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:19.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:18 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:15:19.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:18 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:15:19.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:18 smithi157 ceph-mon[116945]: Adjusting osd_memory_target on smithi157 to 8586M 2023-06-23T01:15:19.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:18 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:19.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:18 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:15:19.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:18 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:15:19.129 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:18 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:19.129 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:18 smithi157 ceph-mon[116945]: osdmap e39: 6 total, 6 up, 6 in 2023-06-23T01:15:19.193 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:18 smithi134 ceph-mon[116146]: purged_snaps scrub starts 2023-06-23T01:15:19.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:18 smithi134 ceph-mon[116146]: purged_snaps scrub ok 2023-06-23T01:15:19.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:18 smithi134 ceph-mon[116146]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-06-23T01:15:19.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:18 smithi134 ceph-mon[116146]: Detected new or changed devices on smithi157 2023-06-23T01:15:19.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:18 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:19.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:18 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:19.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:18 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:15:19.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:18 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:15:19.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:18 smithi134 ceph-mon[116146]: Adjusting osd_memory_target on smithi157 to 8586M 2023-06-23T01:15:19.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:18 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:19.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:18 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:15:19.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:18 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:15:19.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:18 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:19.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:18 smithi134 ceph-mon[116146]: osdmap e39: 6 total, 6 up, 6 in 2023-06-23T01:15:19.226 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:18 smithi120 ceph-mon[79813]: purged_snaps scrub starts 2023-06-23T01:15:19.227 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:18 smithi120 ceph-mon[79813]: purged_snaps scrub ok 2023-06-23T01:15:19.227 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:18 smithi120 ceph-mon[79813]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-06-23T01:15:19.227 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:18 smithi120 ceph-mon[79813]: Detected new or changed devices on smithi157 2023-06-23T01:15:19.227 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:18 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:19.227 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:18 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:19.227 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:18 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:15:19.227 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:18 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:15:19.228 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:18 smithi120 ceph-mon[79813]: Adjusting osd_memory_target on smithi157 to 8586M 2023-06-23T01:15:19.228 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:18 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:19.228 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:18 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:15:19.228 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:18 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:15:19.228 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:18 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:19.228 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:18 smithi120 ceph-mon[79813]: osdmap e39: 6 total, 6 up, 6 in 2023-06-23T01:15:20.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:19 smithi120 ceph-mon[79813]: osdmap e40: 6 total, 6 up, 6 in 2023-06-23T01:15:20.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:19 smithi157 ceph-mon[116945]: osdmap e40: 6 total, 6 up, 6 in 2023-06-23T01:15:20.379 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.376+0000 7f0fd1442700 1 -- 172.21.15.120:0/389999627 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fcc102990 msgr2=0x7f0fcc104d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:15:20.379 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.376+0000 7f0fd1442700 1 --2- 172.21.15.120:0/389999627 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fcc102990 0x7f0fcc104d70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4009d60 tx=0x7f0fc4004d10 comp rx=0 tx=0).stop 2023-06-23T01:15:20.379 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.376+0000 7f0fd1442700 1 -- 172.21.15.120:0/389999627 shutdown_connections 2023-06-23T01:15:20.379 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.376+0000 7f0fd1442700 1 --2- 172.21.15.120:0/389999627 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fcc102990 0x7f0fcc104d70 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:15:20.380 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.376+0000 7f0fd1442700 1 --2- 172.21.15.120:0/389999627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fcc0fd750 0x7f0fcc0ffb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:15:20.380 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.376+0000 7f0fd1442700 1 --2- 172.21.15.120:0/389999627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fcc100070 0x7f0fcc102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:15:20.380 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.376+0000 7f0fd1442700 1 -- 172.21.15.120:0/389999627 >> 172.21.15.120:0/389999627 conn(0x7f0fcc0f7270 msgr2=0x7f0fcc0f96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:15:20.380 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.376+0000 7f0fd1442700 1 -- 172.21.15.120:0/389999627 shutdown_connections 2023-06-23T01:15:20.380 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.376+0000 7f0fd1442700 1 -- 172.21.15.120:0/389999627 wait complete. 2023-06-23T01:15:20.381 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.377+0000 7f0fd1442700 1 Processor -- start 2023-06-23T01:15:20.382 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.377+0000 7f0fd1442700 1 -- start start 2023-06-23T01:15:20.382 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.377+0000 7f0fd1442700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fcc0fd750 0x7f0fcc10f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:15:20.382 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.377+0000 7f0fd1442700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fcc100070 0x7f0fcc10fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:15:20.382 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.377+0000 7f0fd1442700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fcc102990 0x7f0fcc116dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:15:20.382 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.377+0000 7f0fd1442700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc1173e0 con 0x7f0fcc0fd750 2023-06-23T01:15:20.382 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.377+0000 7f0fd1442700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc117550 con 0x7f0fcc102990 2023-06-23T01:15:20.383 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.377+0000 7f0fd1442700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc117710 con 0x7f0fcc100070 2023-06-23T01:15:20.383 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.377+0000 7f0fcaffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fcc0fd750 0x7f0fcc10f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:15:20.383 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.377+0000 7f0fcb7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fcc102990 0x7f0fcc116dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:15:20.383 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.377+0000 7f0fcb7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fcc102990 0x7f0fcc116dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:48942/0 (socket says 172.21.15.120:48942) 2023-06-23T01:15:20.383 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.377+0000 7f0fcaffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fcc0fd750 0x7f0fcc10f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56376/0 (socket says 172.21.15.120:56376) 2023-06-23T01:15:20.383 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.377+0000 7f0fcb7fe700 1 -- 172.21.15.120:0/2656014805 learned_addr learned my addr 172.21.15.120:0/2656014805 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:15:20.383 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.377+0000 7f0fca7fc700 1 --2- 172.21.15.120:0/2656014805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fcc100070 0x7f0fcc10fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:15:20.383 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.378+0000 7f0fcaffd700 1 -- 172.21.15.120:0/2656014805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fcc100070 msgr2=0x7f0fcc10fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:15:20.384 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.378+0000 7f0fcaffd700 1 --2- 172.21.15.120:0/2656014805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fcc100070 0x7f0fcc10fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:15:20.384 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.378+0000 7f0fcaffd700 1 -- 172.21.15.120:0/2656014805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fcc102990 msgr2=0x7f0fcc116dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:15:20.384 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.378+0000 7f0fcaffd700 1 --2- 172.21.15.120:0/2656014805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fcc102990 0x7f0fcc116dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:15:20.384 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.378+0000 7f0fcaffd700 1 -- 172.21.15.120:0/2656014805 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc4009940 con 0x7f0fcc0fd750 2023-06-23T01:15:20.384 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.378+0000 7f0fcb7fe700 1 --2- 172.21.15.120:0/2656014805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fcc102990 0x7f0fcc116dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T01:15:20.384 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.378+0000 7f0fcaffd700 1 --2- 172.21.15.120:0/2656014805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fcc0fd750 0x7f0fcc10f800 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc009fd0 tx=0x7f0fbc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:15:20.384 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.378+0000 7f0fb37fe700 1 -- 172.21.15.120:0/2656014805 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc0047d0 con 0x7f0fcc0fd750 2023-06-23T01:15:20.385 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.378+0000 7f0fb37fe700 1 -- 172.21.15.120:0/2656014805 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fbc00f040 con 0x7f0fcc0fd750 2023-06-23T01:15:20.385 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.378+0000 7f0fd1442700 1 -- 172.21.15.120:0/2656014805 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fcc117990 con 0x7f0fcc0fd750 2023-06-23T01:15:20.385 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.378+0000 7f0fb37fe700 1 -- 172.21.15.120:0/2656014805 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc00c820 con 0x7f0fcc0fd750 2023-06-23T01:15:20.385 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.378+0000 7f0fd1442700 1 -- 172.21.15.120:0/2656014805 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fcc117f00 con 0x7f0fcc0fd750 2023-06-23T01:15:20.386 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.379+0000 7f0fd1442700 1 -- 172.21.15.120:0/2656014805 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fcc1044c0 con 0x7f0fcc0fd750 2023-06-23T01:15:20.386 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.381+0000 7f0fb37fe700 1 -- 172.21.15.120:0/2656014805 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7f0fbc00cc50 con 0x7f0fcc0fd750 2023-06-23T01:15:20.386 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.382+0000 7f0fb37fe700 1 --2- 172.21.15.120:0/2656014805 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f0fb409b680 0x7f0fb409db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:15:20.386 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.382+0000 7f0fb37fe700 1 -- 172.21.15.120:0/2656014805 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4529+0+0 (secure 0 0 0) 0x7f0fbc0c39c0 con 0x7f0fcc0fd750 2023-06-23T01:15:20.388 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.383+0000 7f0fca7fc700 1 --2- 172.21.15.120:0/2656014805 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f0fb409b680 0x7f0fb409db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:15:20.389 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.383+0000 7f0fca7fc700 1 --2- 172.21.15.120:0/2656014805 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f0fb409b680 0x7f0fb409db30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0004f40 tx=0x7f0fc000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:15:20.390 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.387+0000 7f0fb37fe700 1 -- 172.21.15.120:0/2656014805 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f0fbc04da00 con 0x7f0fcc0fd750 2023-06-23T01:15:20.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:19 smithi134 ceph-mon[116146]: osdmap e40: 6 total, 6 up, 6 in 2023-06-23T01:15:20.588 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:20.585+0000 7f0fd1442700 1 -- 172.21.15.120:0/2656014805 --> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f0fcc05de70 con 0x7f0fb409b680 2023-06-23T01:15:21.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:20 smithi120 ceph-mon[79813]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2023-06-23T01:15:21.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:20 smithi120 ceph-mon[79813]: osdmap e41: 6 total, 6 up, 6 in 2023-06-23T01:15:21.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:20 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T01:15:21.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:20 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T01:15:21.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:20 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:15:21.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:20 smithi157 ceph-mon[116945]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2023-06-23T01:15:21.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:20 smithi157 ceph-mon[116945]: osdmap e41: 6 total, 6 up, 6 in 2023-06-23T01:15:21.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:20 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T01:15:21.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:20 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T01:15:21.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:20 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:15:21.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:20 smithi134 ceph-mon[116146]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2023-06-23T01:15:21.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:20 smithi134 ceph-mon[116146]: osdmap e41: 6 total, 6 up, 6 in 2023-06-23T01:15:21.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:20 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T01:15:21.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:20 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T01:15:21.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:20 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:15:22.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:21 smithi120 ceph-mon[79813]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:15:22.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:21 smithi120 ceph-mon[79813]: osdmap e42: 6 total, 6 up, 6 in 2023-06-23T01:15:22.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:21 smithi120 ceph-mon[79813]: pgmap v143: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-06-23T01:15:22.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:21 smithi157 ceph-mon[116945]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:15:22.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:21 smithi157 ceph-mon[116945]: osdmap e42: 6 total, 6 up, 6 in 2023-06-23T01:15:22.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:21 smithi157 ceph-mon[116945]: pgmap v143: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-06-23T01:15:22.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:21 smithi134 ceph-mon[116146]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:15:22.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:21 smithi134 ceph-mon[116146]: osdmap e42: 6 total, 6 up, 6 in 2023-06-23T01:15:22.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:21 smithi134 ceph-mon[116146]: pgmap v143: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-06-23T01:15:24.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:24 smithi157 ceph-mon[116945]: pgmap v144: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2023-06-23T01:15:24.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:24 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/2972967569' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "338c2c4a-d982-4031-aaaf-be76c9dd74a7"}]: dispatch 2023-06-23T01:15:24.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:24 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/2972967569' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "338c2c4a-d982-4031-aaaf-be76c9dd74a7"}]': finished 2023-06-23T01:15:24.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:24 smithi157 ceph-mon[116945]: osdmap e43: 7 total, 6 up, 7 in 2023-06-23T01:15:24.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:24 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-23T01:15:24.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:24 smithi134 ceph-mon[116146]: pgmap v144: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2023-06-23T01:15:24.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:24 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/2972967569' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "338c2c4a-d982-4031-aaaf-be76c9dd74a7"}]: dispatch 2023-06-23T01:15:24.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:24 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/2972967569' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "338c2c4a-d982-4031-aaaf-be76c9dd74a7"}]': finished 2023-06-23T01:15:24.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:24 smithi134 ceph-mon[116146]: osdmap e43: 7 total, 6 up, 7 in 2023-06-23T01:15:24.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:24 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-23T01:15:24.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:24 smithi120 ceph-mon[79813]: pgmap v144: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2023-06-23T01:15:24.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:24 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/2972967569' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "338c2c4a-d982-4031-aaaf-be76c9dd74a7"}]: dispatch 2023-06-23T01:15:24.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:24 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/2972967569' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "338c2c4a-d982-4031-aaaf-be76c9dd74a7"}]': finished 2023-06-23T01:15:24.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:24 smithi120 ceph-mon[79813]: osdmap e43: 7 total, 6 up, 7 in 2023-06-23T01:15:24.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:24 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-23T01:15:25.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:25 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/2659719737' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T01:15:25.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:25 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/2659719737' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T01:15:25.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:25 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/2659719737' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T01:15:26.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:26 smithi157 ceph-mon[116945]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-06-23T01:15:26.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:26 smithi134 ceph-mon[116146]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-06-23T01:15:26.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:26 smithi120 ceph-mon[79813]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-06-23T01:15:29.227 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:28 smithi120 ceph-mon[79813]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-06-23T01:15:29.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:28 smithi157 ceph-mon[116945]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-06-23T01:15:29.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:28 smithi134 ceph-mon[116146]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-06-23T01:15:31.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:30 smithi120 ceph-mon[79813]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-06-23T01:15:31.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:30 smithi157 ceph-mon[116945]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-06-23T01:15:31.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:30 smithi134 ceph-mon[116146]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-06-23T01:15:32.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:31 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-06-23T01:15:32.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:31 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:15:32.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:31 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-06-23T01:15:32.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:31 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:15:32.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:31 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-06-23T01:15:32.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:31 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:15:33.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:32 smithi120 ceph-mon[79813]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-06-23T01:15:33.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:32 smithi120 ceph-mon[79813]: Deploying daemon osd.6 on smithi120 2023-06-23T01:15:33.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:32 smithi157 ceph-mon[116945]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-06-23T01:15:33.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:32 smithi157 ceph-mon[116945]: Deploying daemon osd.6 on smithi120 2023-06-23T01:15:33.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:32 smithi134 ceph-mon[116146]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-06-23T01:15:33.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:32 smithi134 ceph-mon[116146]: Deploying daemon osd.6 on smithi120 2023-06-23T01:15:35.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:34 smithi120 ceph-mon[79813]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-06-23T01:15:35.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:34 smithi157 ceph-mon[116945]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-06-23T01:15:35.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:34 smithi134 ceph-mon[116146]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-06-23T01:15:37.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:36 smithi120 ceph-mon[79813]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-06-23T01:15:37.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:36 smithi157 ceph-mon[116945]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-06-23T01:15:37.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:36 smithi134 ceph-mon[116146]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-06-23T01:15:38.357 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:38 smithi120 ceph-mon[79813]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-06-23T01:15:38.357 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:38 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:38.357 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:38 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:15:38.357 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:38 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:38.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:38 smithi157 ceph-mon[116945]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-06-23T01:15:38.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:38 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:38.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:38 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:15:38.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:38 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:38.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:38 smithi134 ceph-mon[116146]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-06-23T01:15:38.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:38 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:38.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:38 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:15:38.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:38 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:40.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:40 smithi157 ceph-mon[116945]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-06-23T01:15:40.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:40 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:40.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:40 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:40.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:40 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:15:40.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:40 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:15:40.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:40 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:40.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:40 smithi134 ceph-mon[116146]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-06-23T01:15:40.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:40 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:40.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:40 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:40.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:40 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:15:40.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:40 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:15:40.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:40 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:40.448 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:40.445+0000 7f0fb37fe700 1 -- 172.21.15.120:0/2656014805 <== mgr.14152 v2:172.21.15.120:6800/1770912857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0fcc05de70 con 0x7f0fb409b680 2023-06-23T01:15:40.449 INFO:teuthology.orchestra.run.smithi120.stdout:Created osd(s) 6 on host 'smithi120' 2023-06-23T01:15:40.451 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:40.448+0000 7f0fd1442700 1 -- 172.21.15.120:0/2656014805 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f0fb409b680 msgr2=0x7f0fb409db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:15:40.451 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:40.448+0000 7f0fd1442700 1 --2- 172.21.15.120:0/2656014805 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f0fb409b680 0x7f0fb409db30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0004f40 tx=0x7f0fc000b040 comp rx=0 tx=0).stop 2023-06-23T01:15:40.451 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:40.448+0000 7f0fd1442700 1 -- 172.21.15.120:0/2656014805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fcc0fd750 msgr2=0x7f0fcc10f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:15:40.452 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:40.448+0000 7f0fd1442700 1 --2- 172.21.15.120:0/2656014805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fcc0fd750 0x7f0fcc10f800 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc009fd0 tx=0x7f0fbc00b040 comp rx=0 tx=0).stop 2023-06-23T01:15:40.452 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:40.448+0000 7f0fd1442700 1 -- 172.21.15.120:0/2656014805 shutdown_connections 2023-06-23T01:15:40.452 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:40.448+0000 7f0fd1442700 1 --2- 172.21.15.120:0/2656014805 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f0fb409b680 0x7f0fb409db30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:15:40.453 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:40.448+0000 7f0fd1442700 1 --2- 172.21.15.120:0/2656014805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fcc0fd750 0x7f0fcc10f800 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:15:40.453 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:40.448+0000 7f0fd1442700 1 --2- 172.21.15.120:0/2656014805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fcc102990 0x7f0fcc116dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:15:40.453 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:40.448+0000 7f0fd1442700 1 --2- 172.21.15.120:0/2656014805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fcc100070 0x7f0fcc10fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:15:40.453 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:40.448+0000 7f0fd1442700 1 -- 172.21.15.120:0/2656014805 >> 172.21.15.120:0/2656014805 conn(0x7f0fcc0f7270 msgr2=0x7f0fcc0fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:15:40.454 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:40.448+0000 7f0fd1442700 1 -- 172.21.15.120:0/2656014805 shutdown_connections 2023-06-23T01:15:40.454 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:15:40.448+0000 7f0fd1442700 1 -- 172.21.15.120:0/2656014805 wait complete. 2023-06-23T01:15:40.472 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:40 smithi120 ceph-mon[79813]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-06-23T01:15:40.472 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:40 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:40.472 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:40 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:40.472 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:40 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:15:40.472 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:40 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:15:40.473 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:40 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:40.875 DEBUG:teuthology.orchestra.run.smithi120:osd.6> sudo journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@osd.6.service 2023-06-23T01:15:40.879 INFO:tasks.cephadm:Deploying osd.7 on smithi134 with /dev/vg_nvme/lv_2... 2023-06-23T01:15:40.880 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-06-23T01:15:40.943 INFO:journalctl@ceph.osd.6.smithi120.stdout:-- Logs begin at Fri 2023-06-23 00:53:53 UTC. -- 2023-06-23T01:15:41.444 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:15:41 smithi120 conmon[110061]: 2023-06-23T01:15:41.049+0000 7f8bda39b540 -1 osd.6 0 log_to_monitors true 2023-06-23T01:15:41.467 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.b/config 2023-06-23T01:15:41.696 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:41 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:41.696 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:41 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:41.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:41 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:15:41.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:41 smithi134 ceph-mon[116146]: from='osd.6 [v2:172.21.15.120:6818/2097345225,v1:172.21.15.120:6819/2097345225]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-06-23T01:15:41.739 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:41 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:41.740 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:41 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:41.740 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:41 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:15:41.740 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:41 smithi120 ceph-mon[79813]: from='osd.6 [v2:172.21.15.120:6818/2097345225,v1:172.21.15.120:6819/2097345225]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-06-23T01:15:41.876 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:41 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:41.876 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:41 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:41.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:41 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:15:41.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:41 smithi157 ceph-mon[116945]: from='osd.6 [v2:172.21.15.120:6818/2097345225,v1:172.21.15.120:6819/2097345225]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-06-23T01:15:42.694 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:42 smithi134 ceph-mon[116146]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-06-23T01:15:42.695 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:42 smithi134 ceph-mon[116146]: from='osd.6 [v2:172.21.15.120:6818/2097345225,v1:172.21.15.120:6819/2097345225]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-06-23T01:15:42.695 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:42 smithi134 ceph-mon[116146]: osdmap e44: 7 total, 6 up, 7 in 2023-06-23T01:15:42.695 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:42 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-23T01:15:42.695 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:42 smithi134 ceph-mon[116146]: from='osd.6 [v2:172.21.15.120:6818/2097345225,v1:172.21.15.120:6819/2097345225]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2023-06-23T01:15:42.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:42 smithi120 ceph-mon[79813]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-06-23T01:15:42.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:42 smithi120 ceph-mon[79813]: from='osd.6 [v2:172.21.15.120:6818/2097345225,v1:172.21.15.120:6819/2097345225]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-06-23T01:15:42.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:42 smithi120 ceph-mon[79813]: osdmap e44: 7 total, 6 up, 7 in 2023-06-23T01:15:42.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:42 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-23T01:15:42.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:42 smithi120 ceph-mon[79813]: from='osd.6 [v2:172.21.15.120:6818/2097345225,v1:172.21.15.120:6819/2097345225]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2023-06-23T01:15:42.762 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:15:42 smithi120 conmon[110061]: 2023-06-23T01:15:42.467+0000 7f8bd0b80700 -1 osd.6 0 waiting for initial osdmap 2023-06-23T01:15:42.762 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:15:42 smithi120 conmon[110061]: 2023-06-23T01:15:42.483+0000 7f8bcb97b700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T01:15:42.876 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:42 smithi157 ceph-mon[116945]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-06-23T01:15:42.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:42 smithi157 ceph-mon[116945]: from='osd.6 [v2:172.21.15.120:6818/2097345225,v1:172.21.15.120:6819/2097345225]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-06-23T01:15:42.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:42 smithi157 ceph-mon[116945]: osdmap e44: 7 total, 6 up, 7 in 2023-06-23T01:15:42.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:42 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-23T01:15:42.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:42 smithi157 ceph-mon[116945]: from='osd.6 [v2:172.21.15.120:6818/2097345225,v1:172.21.15.120:6819/2097345225]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2023-06-23T01:15:43.216 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-06-23T01:15:43.217 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-06-23T01:15:43.217 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2023-06-23T01:15:43.217 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2023-06-23T01:15:43.217 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0109335 s, 959 MB/s 2023-06-23T01:15:43.217 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2023-06-23T01:15:43.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:43 smithi120 ceph-mon[79813]: from='osd.6 [v2:172.21.15.120:6818/2097345225,v1:172.21.15.120:6819/2097345225]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2023-06-23T01:15:43.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:43 smithi120 ceph-mon[79813]: osdmap e45: 7 total, 6 up, 7 in 2023-06-23T01:15:43.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:43 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-23T01:15:43.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:43 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-23T01:15:43.694 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:43 smithi134 ceph-mon[116146]: from='osd.6 [v2:172.21.15.120:6818/2097345225,v1:172.21.15.120:6819/2097345225]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2023-06-23T01:15:43.695 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:43 smithi134 ceph-mon[116146]: osdmap e45: 7 total, 6 up, 7 in 2023-06-23T01:15:43.695 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:43 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-23T01:15:43.695 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:43 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-23T01:15:43.876 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:43 smithi157 ceph-mon[116945]: from='osd.6 [v2:172.21.15.120:6818/2097345225,v1:172.21.15.120:6819/2097345225]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2023-06-23T01:15:43.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:43 smithi157 ceph-mon[116945]: osdmap e45: 7 total, 6 up, 7 in 2023-06-23T01:15:43.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:43 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-23T01:15:43.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:43 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-23T01:15:43.897 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph orch daemon add osd smithi134:vg_nvme/lv_2 2023-06-23T01:15:44.473 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.b/config 2023-06-23T01:15:44.732 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:44 smithi134 ceph-mon[116146]: purged_snaps scrub starts 2023-06-23T01:15:44.733 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:44 smithi134 ceph-mon[116146]: purged_snaps scrub ok 2023-06-23T01:15:44.733 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:44 smithi134 ceph-mon[116146]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-06-23T01:15:44.733 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:44 smithi134 ceph-mon[116146]: osd.6 [v2:172.21.15.120:6818/2097345225,v1:172.21.15.120:6819/2097345225] boot 2023-06-23T01:15:44.733 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:44 smithi134 ceph-mon[116146]: osdmap e46: 7 total, 7 up, 7 in 2023-06-23T01:15:44.733 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:44 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-23T01:15:44.733 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:44 smithi134 ceph-mon[116146]: Detected new or changed devices on smithi120 2023-06-23T01:15:44.734 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:44 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:44.734 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:44 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:44.734 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:44 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:15:44.734 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:44 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:15:44.734 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:44 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:15:44.734 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:44 smithi134 ceph-mon[116146]: Adjusting osd_memory_target on smithi120 to 5755M 2023-06-23T01:15:44.734 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:44 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:44.735 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:44 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:15:44.735 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:44 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:15:44.735 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:44 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:44.760 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:44 smithi120 ceph-mon[79813]: purged_snaps scrub starts 2023-06-23T01:15:44.760 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:44 smithi120 ceph-mon[79813]: purged_snaps scrub ok 2023-06-23T01:15:44.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:44 smithi120 ceph-mon[79813]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-06-23T01:15:44.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:44 smithi120 ceph-mon[79813]: osd.6 [v2:172.21.15.120:6818/2097345225,v1:172.21.15.120:6819/2097345225] boot 2023-06-23T01:15:44.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:44 smithi120 ceph-mon[79813]: osdmap e46: 7 total, 7 up, 7 in 2023-06-23T01:15:44.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:44 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-23T01:15:44.762 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:44 smithi120 ceph-mon[79813]: Detected new or changed devices on smithi120 2023-06-23T01:15:44.762 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:44 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:44.762 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:44 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:44.762 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:44 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:15:44.762 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:44 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:15:44.763 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:44 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:15:44.763 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:44 smithi120 ceph-mon[79813]: Adjusting osd_memory_target on smithi120 to 5755M 2023-06-23T01:15:44.763 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:44 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:44.763 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:44 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:15:44.763 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:44 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:15:44.763 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:44 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:44.876 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:44 smithi157 ceph-mon[116945]: purged_snaps scrub starts 2023-06-23T01:15:44.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:44 smithi157 ceph-mon[116945]: purged_snaps scrub ok 2023-06-23T01:15:44.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:44 smithi157 ceph-mon[116945]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-06-23T01:15:44.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:44 smithi157 ceph-mon[116945]: osd.6 [v2:172.21.15.120:6818/2097345225,v1:172.21.15.120:6819/2097345225] boot 2023-06-23T01:15:44.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:44 smithi157 ceph-mon[116945]: osdmap e46: 7 total, 7 up, 7 in 2023-06-23T01:15:44.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:44 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-23T01:15:44.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:44 smithi157 ceph-mon[116945]: Detected new or changed devices on smithi120 2023-06-23T01:15:44.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:44 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:44.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:44 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:44.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:44 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:15:44.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:44 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:15:44.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:44 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:15:44.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:44 smithi157 ceph-mon[116945]: Adjusting osd_memory_target on smithi120 to 5755M 2023-06-23T01:15:44.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:44 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:44.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:44 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:15:44.879 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:44 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:15:44.879 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:44 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:15:45.693 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:45 smithi134 ceph-mon[116146]: osdmap e47: 7 total, 7 up, 7 in 2023-06-23T01:15:45.855 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.850+0000 7f84a8f55700 1 -- 172.21.15.134:0/4167343674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84a40fe820 msgr2=0x7f84a41050e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:15:45.856 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.850+0000 7f84a8f55700 1 --2- 172.21.15.134:0/4167343674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84a40fe820 0x7f84a41050e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f849c009d60 tx=0x7f849c004d10 comp rx=0 tx=0).stop 2023-06-23T01:15:45.856 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.850+0000 7f84a8f55700 1 -- 172.21.15.134:0/4167343674 shutdown_connections 2023-06-23T01:15:45.856 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.850+0000 7f84a8f55700 1 --2- 172.21.15.134:0/4167343674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84a40fe820 0x7f84a41050e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:15:45.856 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.850+0000 7f84a8f55700 1 --2- 172.21.15.134:0/4167343674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84a40fcc80 0x7f84a40fd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:15:45.856 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.850+0000 7f84a8f55700 1 --2- 172.21.15.134:0/4167343674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84a40fde90 0x7f84a40fe2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:15:45.857 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.850+0000 7f84a8f55700 1 -- 172.21.15.134:0/4167343674 >> 172.21.15.134:0/4167343674 conn(0x7f84a40f83c0 msgr2=0x7f84a40fa810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:15:45.857 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.850+0000 7f84a8f55700 1 -- 172.21.15.134:0/4167343674 shutdown_connections 2023-06-23T01:15:45.857 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.850+0000 7f84a8f55700 1 -- 172.21.15.134:0/4167343674 wait complete. 2023-06-23T01:15:45.857 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.851+0000 7f84a8f55700 1 Processor -- start 2023-06-23T01:15:45.857 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.851+0000 7f84a8f55700 1 -- start start 2023-06-23T01:15:45.857 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.851+0000 7f84a8f55700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84a40fcc80 0x7f84a4071720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:15:45.857 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.851+0000 7f84a8f55700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84a40fde90 0x7f84a406fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:15:45.857 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.851+0000 7f84a8f55700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84a40fe820 0x7f84a40702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:15:45.858 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.851+0000 7f84a8f55700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a4070a30 con 0x7f84a40fcc80 2023-06-23T01:15:45.858 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.851+0000 7f84a8f55700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a4070ba0 con 0x7f84a40fe820 2023-06-23T01:15:45.858 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.851+0000 7f84a8f55700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a4117950 con 0x7f84a40fde90 2023-06-23T01:15:45.858 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.851+0000 7f84a2d9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84a40fe820 0x7f84a40702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:15:45.858 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.851+0000 7f84a2d9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84a40fe820 0x7f84a40702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:43294/0 (socket says 172.21.15.134:43294) 2023-06-23T01:15:45.858 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.851+0000 7f84a2d9d700 1 -- 172.21.15.134:0/335273060 learned_addr learned my addr 172.21.15.134:0/335273060 (peer_addr_for_me v2:172.21.15.134:0/0) 2023-06-23T01:15:45.858 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.851+0000 7f84a259c700 1 --2- 172.21.15.134:0/335273060 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84a40fcc80 0x7f84a4071720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:15:45.858 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.851+0000 7f84a1d9b700 1 --2- 172.21.15.134:0/335273060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84a40fde90 0x7f84a406fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:15:45.859 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.851+0000 7f84a2d9d700 1 -- 172.21.15.134:0/335273060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84a40fde90 msgr2=0x7f84a406fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:15:45.859 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.851+0000 7f84a2d9d700 1 --2- 172.21.15.134:0/335273060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84a40fde90 0x7f84a406fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:15:45.859 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.851+0000 7f84a2d9d700 1 -- 172.21.15.134:0/335273060 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84a40fcc80 msgr2=0x7f84a4071720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:15:45.859 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.851+0000 7f84a2d9d700 1 --2- 172.21.15.134:0/335273060 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84a40fcc80 0x7f84a4071720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:15:45.859 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.851+0000 7f84a2d9d700 1 -- 172.21.15.134:0/335273060 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f849c009940 con 0x7f84a40fe820 2023-06-23T01:15:45.859 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.852+0000 7f84a2d9d700 1 --2- 172.21.15.134:0/335273060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84a40fe820 0x7f84a40702c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f849c0091e0 tx=0x7f849c004d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:15:45.860 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.852+0000 7f8492ffd700 1 -- 172.21.15.134:0/335273060 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849c00e070 con 0x7f84a40fe820 2023-06-23T01:15:45.860 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.852+0000 7f84a8f55700 1 -- 172.21.15.134:0/335273060 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84a4117ba0 con 0x7f84a40fe820 2023-06-23T01:15:45.860 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.852+0000 7f8492ffd700 1 -- 172.21.15.134:0/335273060 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f849c012d00 con 0x7f84a40fe820 2023-06-23T01:15:45.860 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.852+0000 7f8492ffd700 1 -- 172.21.15.134:0/335273060 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849c01cc90 con 0x7f84a40fe820 2023-06-23T01:15:45.860 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.852+0000 7f84a8f55700 1 -- 172.21.15.134:0/335273060 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84a4117fb0 con 0x7f84a40fe820 2023-06-23T01:15:45.861 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.853+0000 7f84a8f55700 1 -- 172.21.15.134:0/335273060 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84a4108520 con 0x7f84a40fe820 2023-06-23T01:15:45.861 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.854+0000 7f8492ffd700 1 -- 172.21.15.134:0/335273060 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7f849c01c440 con 0x7f84a40fe820 2023-06-23T01:15:45.861 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.854+0000 7f8492ffd700 1 --2- 172.21.15.134:0/335273060 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f848c09b680 0x7f848c09db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:15:45.861 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.854+0000 7f8492ffd700 1 -- 172.21.15.134:0/335273060 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4961+0+0 (secure 0 0 0) 0x7f849c0c6c50 con 0x7f84a40fe820 2023-06-23T01:15:45.862 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.855+0000 7f84a259c700 1 --2- 172.21.15.134:0/335273060 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f848c09b680 0x7f848c09db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:15:45.862 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.855+0000 7f84a259c700 1 --2- 172.21.15.134:0/335273060 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f848c09b680 0x7f848c09db30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8494006fd0 tx=0x7f8494008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:15:45.862 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:45.858+0000 7f8492ffd700 1 -- 172.21.15.134:0/335273060 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f849c0931c0 con 0x7f84a40fe820 2023-06-23T01:15:46.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:45 smithi120 ceph-mon[79813]: osdmap e47: 7 total, 7 up, 7 in 2023-06-23T01:15:46.049 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:15:46.046+0000 7f84a8f55700 1 -- 172.21.15.134:0/335273060 --> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f84a405de70 con 0x7f848c09b680 2023-06-23T01:15:46.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:45 smithi157 ceph-mon[116945]: osdmap e47: 7 total, 7 up, 7 in 2023-06-23T01:15:46.903 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:46 smithi134 ceph-mon[116146]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2023-06-23T01:15:46.903 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:46 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T01:15:46.903 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:46 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T01:15:46.903 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:46 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:15:47.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:46 smithi120 ceph-mon[79813]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2023-06-23T01:15:47.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:46 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T01:15:47.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:46 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T01:15:47.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:46 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:15:47.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:46 smithi157 ceph-mon[116945]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2023-06-23T01:15:47.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:46 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T01:15:47.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:46 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T01:15:47.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:46 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:15:48.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:47 smithi120 ceph-mon[79813]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:15:48.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:47 smithi157 ceph-mon[116945]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:15:48.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:47 smithi134 ceph-mon[116146]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:15:49.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:48 smithi120 ceph-mon[79813]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2023-06-23T01:15:49.304 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:48 smithi120 ceph-mon[79813]: from='client.? 172.21.15.134:0/2040970224' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bff92799-ae9c-4409-be65-7ffc27a1c3f0"}]: dispatch 2023-06-23T01:15:49.305 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:48 smithi120 ceph-mon[79813]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bff92799-ae9c-4409-be65-7ffc27a1c3f0"}]: dispatch 2023-06-23T01:15:49.305 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:48 smithi120 ceph-mon[79813]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bff92799-ae9c-4409-be65-7ffc27a1c3f0"}]': finished 2023-06-23T01:15:49.305 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:48 smithi120 ceph-mon[79813]: osdmap e48: 8 total, 7 up, 8 in 2023-06-23T01:15:49.306 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:48 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-23T01:15:49.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:48 smithi157 ceph-mon[116945]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2023-06-23T01:15:49.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:48 smithi157 ceph-mon[116945]: from='client.? 172.21.15.134:0/2040970224' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bff92799-ae9c-4409-be65-7ffc27a1c3f0"}]: dispatch 2023-06-23T01:15:49.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:48 smithi157 ceph-mon[116945]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bff92799-ae9c-4409-be65-7ffc27a1c3f0"}]: dispatch 2023-06-23T01:15:49.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:48 smithi157 ceph-mon[116945]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bff92799-ae9c-4409-be65-7ffc27a1c3f0"}]': finished 2023-06-23T01:15:49.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:48 smithi157 ceph-mon[116945]: osdmap e48: 8 total, 7 up, 8 in 2023-06-23T01:15:49.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:48 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-23T01:15:49.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:48 smithi134 ceph-mon[116146]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2023-06-23T01:15:49.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:48 smithi134 ceph-mon[116146]: from='client.? 172.21.15.134:0/2040970224' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bff92799-ae9c-4409-be65-7ffc27a1c3f0"}]: dispatch 2023-06-23T01:15:49.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:48 smithi134 ceph-mon[116146]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bff92799-ae9c-4409-be65-7ffc27a1c3f0"}]: dispatch 2023-06-23T01:15:49.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:48 smithi134 ceph-mon[116146]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bff92799-ae9c-4409-be65-7ffc27a1c3f0"}]': finished 2023-06-23T01:15:49.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:48 smithi134 ceph-mon[116146]: osdmap e48: 8 total, 7 up, 8 in 2023-06-23T01:15:49.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:48 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-23T01:15:50.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:49 smithi120 ceph-mon[79813]: from='client.? 172.21.15.134:0/157340018' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T01:15:50.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:49 smithi157 ceph-mon[116945]: from='client.? 172.21.15.134:0/157340018' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T01:15:50.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:49 smithi134 ceph-mon[116146]: from='client.? 172.21.15.134:0/157340018' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T01:15:51.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:50 smithi120 ceph-mon[79813]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2023-06-23T01:15:51.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:50 smithi157 ceph-mon[116945]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2023-06-23T01:15:51.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:50 smithi134 ceph-mon[116146]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2023-06-23T01:15:53.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:52 smithi120 ceph-mon[79813]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-06-23T01:15:53.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:52 smithi157 ceph-mon[116945]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-06-23T01:15:53.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:52 smithi134 ceph-mon[116146]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-06-23T01:15:55.089 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:54 smithi134 ceph-mon[116146]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-06-23T01:15:55.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:54 smithi120 ceph-mon[79813]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-06-23T01:15:55.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:54 smithi157 ceph-mon[116945]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-06-23T01:15:57.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:56 smithi120 ceph-mon[79813]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-06-23T01:15:57.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:56 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-06-23T01:15:57.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:56 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:15:57.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:56 smithi157 ceph-mon[116945]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-06-23T01:15:57.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:56 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-06-23T01:15:57.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:56 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:15:57.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:56 smithi134 ceph-mon[116146]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-06-23T01:15:57.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:56 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-06-23T01:15:57.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:56 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:15:58.103 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:57 smithi134 ceph-mon[116146]: Deploying daemon osd.7 on smithi134 2023-06-23T01:15:58.104 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:15:57 smithi134 ceph-mon[116146]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-06-23T01:15:58.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:57 smithi120 ceph-mon[79813]: Deploying daemon osd.7 on smithi134 2023-06-23T01:15:58.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:15:57 smithi120 ceph-mon[79813]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-06-23T01:15:58.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:57 smithi157 ceph-mon[116945]: Deploying daemon osd.7 on smithi134 2023-06-23T01:15:58.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:15:57 smithi157 ceph-mon[116945]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-06-23T01:16:00.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:00 smithi157 ceph-mon[116945]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-06-23T01:16:00.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:00 smithi134 ceph-mon[116146]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-06-23T01:16:00.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:00 smithi120 ceph-mon[79813]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-06-23T01:16:02.359 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:02 smithi134 ceph-mon[116146]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-06-23T01:16:02.359 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:02 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:02.359 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:02 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:02.360 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:02 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:16:02.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:02 smithi120 ceph-mon[79813]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-06-23T01:16:02.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:02 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:02.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:02 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:02.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:02 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:16:02.626 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:02 smithi157 ceph-mon[116945]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-06-23T01:16:02.627 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:02 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:02.627 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:02 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:02.627 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:02 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:16:04.519 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:16:04.515+0000 7f8492ffd700 1 -- 172.21.15.134:0/335273060 <== mgr.14152 v2:172.21.15.120:6800/1770912857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f84a405de70 con 0x7f848c09b680 2023-06-23T01:16:04.519 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 7 on host 'smithi134' 2023-06-23T01:16:04.521 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:16:04.517+0000 7f84a8f55700 1 -- 172.21.15.134:0/335273060 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f848c09b680 msgr2=0x7f848c09db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:16:04.521 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:16:04.517+0000 7f84a8f55700 1 --2- 172.21.15.134:0/335273060 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f848c09b680 0x7f848c09db30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8494006fd0 tx=0x7f8494008040 comp rx=0 tx=0).stop 2023-06-23T01:16:04.521 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:16:04.517+0000 7f84a8f55700 1 -- 172.21.15.134:0/335273060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84a40fe820 msgr2=0x7f84a40702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:16:04.521 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:16:04.517+0000 7f84a8f55700 1 --2- 172.21.15.134:0/335273060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84a40fe820 0x7f84a40702c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f849c0091e0 tx=0x7f849c004d10 comp rx=0 tx=0).stop 2023-06-23T01:16:04.522 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:16:04.517+0000 7f84a8f55700 1 -- 172.21.15.134:0/335273060 shutdown_connections 2023-06-23T01:16:04.522 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:16:04.517+0000 7f84a8f55700 1 --2- 172.21.15.134:0/335273060 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f848c09b680 0x7f848c09db30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:16:04.522 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:16:04.517+0000 7f84a8f55700 1 --2- 172.21.15.134:0/335273060 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84a40fcc80 0x7f84a4071720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:16:04.522 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:16:04.517+0000 7f84a8f55700 1 --2- 172.21.15.134:0/335273060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84a40fe820 0x7f84a40702c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:16:04.522 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:16:04.517+0000 7f84a8f55700 1 --2- 172.21.15.134:0/335273060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84a40fde90 0x7f84a406fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:16:04.522 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:16:04.517+0000 7f84a8f55700 1 -- 172.21.15.134:0/335273060 >> 172.21.15.134:0/335273060 conn(0x7f84a40f83c0 msgr2=0x7f84a40fa480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:16:04.522 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:16:04.518+0000 7f84a8f55700 1 -- 172.21.15.134:0/335273060 shutdown_connections 2023-06-23T01:16:04.523 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:16:04.518+0000 7f84a8f55700 1 -- 172.21.15.134:0/335273060 wait complete. 2023-06-23T01:16:05.113 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:04 smithi134 ceph-mon[116146]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-06-23T01:16:05.114 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:04 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:05.114 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:04 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:05.114 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:04 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:16:05.115 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:04 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:16:05.115 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:04 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:05.115 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:04 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:05.115 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:04 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:05.115 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:04 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:16:05.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:04 smithi157 ceph-mon[116945]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-06-23T01:16:05.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:04 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:05.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:04 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:05.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:04 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:16:05.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:04 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:16:05.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:04 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:05.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:04 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:05.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:04 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:05.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:04 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:16:05.129 DEBUG:teuthology.orchestra.run.smithi134:osd.7> sudo journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@osd.7.service 2023-06-23T01:16:05.131 INFO:tasks.cephadm:Deploying osd.8 on smithi157 with /dev/vg_nvme/lv_2... 2023-06-23T01:16:05.132 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-06-23T01:16:05.158 INFO:journalctl@ceph.osd.7.smithi134.stdout:-- Logs begin at Fri 2023-06-23 00:49:19 UTC. -- 2023-06-23T01:16:05.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:04 smithi120 ceph-mon[79813]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-06-23T01:16:05.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:04 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:05.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:04 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:05.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:04 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:16:05.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:04 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:16:05.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:04 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:05.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:04 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:05.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:04 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:05.263 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:04 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:16:05.443 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:16:05 smithi134 conmon[136657]: 2023-06-23T01:16:05.176+0000 7fddf91d4540 -1 osd.7 0 log_to_monitors true 2023-06-23T01:16:05.722 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.c/config 2023-06-23T01:16:05.854 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:05 smithi134 ceph-mon[116146]: from='osd.7 [v2:172.21.15.134:6816/1345745298,v1:172.21.15.134:6817/1345745298]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-06-23T01:16:05.855 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:05 smithi134 ceph-mon[116146]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-06-23T01:16:06.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:05 smithi157 ceph-mon[116945]: from='osd.7 [v2:172.21.15.134:6816/1345745298,v1:172.21.15.134:6817/1345745298]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-06-23T01:16:06.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:05 smithi157 ceph-mon[116945]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-06-23T01:16:06.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:05 smithi120 ceph-mon[79813]: from='osd.7 [v2:172.21.15.134:6816/1345745298,v1:172.21.15.134:6817/1345745298]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-06-23T01:16:06.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:05 smithi120 ceph-mon[79813]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-06-23T01:16:06.895 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:06 smithi157 ceph-mon[116945]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-06-23T01:16:06.895 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:06 smithi157 ceph-mon[116945]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-06-23T01:16:06.895 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:06 smithi157 ceph-mon[116945]: osdmap e49: 8 total, 7 up, 8 in 2023-06-23T01:16:06.895 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:06 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-23T01:16:06.895 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:06 smithi157 ceph-mon[116945]: from='osd.7 [v2:172.21.15.134:6816/1345745298,v1:172.21.15.134:6817/1345745298]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-06-23T01:16:06.895 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:06 smithi157 ceph-mon[116945]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-06-23T01:16:07.182 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:16:06 smithi134 conmon[136657]: 2023-06-23T01:16:06.866+0000 7fddef9b9700 -1 osd.7 0 waiting for initial osdmap 2023-06-23T01:16:07.183 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:16:06 smithi134 conmon[136657]: 2023-06-23T01:16:06.878+0000 7fdde87b0700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T01:16:07.183 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:06 smithi134 ceph-mon[116146]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-06-23T01:16:07.183 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:06 smithi134 ceph-mon[116146]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-06-23T01:16:07.183 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:06 smithi134 ceph-mon[116146]: osdmap e49: 8 total, 7 up, 8 in 2023-06-23T01:16:07.183 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:06 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-23T01:16:07.184 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:06 smithi134 ceph-mon[116146]: from='osd.7 [v2:172.21.15.134:6816/1345745298,v1:172.21.15.134:6817/1345745298]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-06-23T01:16:07.184 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:06 smithi134 ceph-mon[116146]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-06-23T01:16:07.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:06 smithi120 ceph-mon[79813]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-06-23T01:16:07.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:06 smithi120 ceph-mon[79813]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-06-23T01:16:07.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:06 smithi120 ceph-mon[79813]: osdmap e49: 8 total, 7 up, 8 in 2023-06-23T01:16:07.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:06 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-23T01:16:07.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:06 smithi120 ceph-mon[79813]: from='osd.7 [v2:172.21.15.134:6816/1345745298,v1:172.21.15.134:6817/1345745298]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-06-23T01:16:07.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:06 smithi120 ceph-mon[79813]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-06-23T01:16:07.365 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-06-23T01:16:07.365 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-06-23T01:16:07.365 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2023-06-23T01:16:07.365 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2023-06-23T01:16:07.365 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0122027 s, 859 MB/s 2023-06-23T01:16:07.366 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2023-06-23T01:16:07.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:07 smithi134 ceph-mon[116146]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-06-23T01:16:07.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:07 smithi134 ceph-mon[116146]: osdmap e50: 8 total, 7 up, 8 in 2023-06-23T01:16:07.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:07 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-23T01:16:07.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:07 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-23T01:16:07.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:07 smithi134 ceph-mon[116146]: osd.7 [v2:172.21.15.134:6816/1345745298,v1:172.21.15.134:6817/1345745298] boot 2023-06-23T01:16:07.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:07 smithi134 ceph-mon[116146]: osdmap e51: 8 total, 8 up, 8 in 2023-06-23T01:16:07.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:07 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-23T01:16:07.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:07 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:07.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:07 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:07.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:07 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:16:07.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:07 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:16:07.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:07 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:16:07.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:07 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:07.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:07 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:16:07.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:07 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:16:07.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:07 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:08.051 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph orch daemon add osd smithi157:vg_nvme/lv_2 2023-06-23T01:16:08.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:07 smithi157 ceph-mon[116945]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-06-23T01:16:08.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:07 smithi157 ceph-mon[116945]: osdmap e50: 8 total, 7 up, 8 in 2023-06-23T01:16:08.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:07 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-23T01:16:08.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:07 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-23T01:16:08.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:07 smithi157 ceph-mon[116945]: osd.7 [v2:172.21.15.134:6816/1345745298,v1:172.21.15.134:6817/1345745298] boot 2023-06-23T01:16:08.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:07 smithi157 ceph-mon[116945]: osdmap e51: 8 total, 8 up, 8 in 2023-06-23T01:16:08.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:07 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-23T01:16:08.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:07 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:08.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:07 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:08.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:07 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:16:08.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:07 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:16:08.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:07 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:16:08.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:07 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:08.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:07 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:16:08.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:07 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:16:08.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:07 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:08.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:07 smithi120 ceph-mon[79813]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-06-23T01:16:08.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:07 smithi120 ceph-mon[79813]: osdmap e50: 8 total, 7 up, 8 in 2023-06-23T01:16:08.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:07 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-23T01:16:08.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:07 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-23T01:16:08.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:07 smithi120 ceph-mon[79813]: osd.7 [v2:172.21.15.134:6816/1345745298,v1:172.21.15.134:6817/1345745298] boot 2023-06-23T01:16:08.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:07 smithi120 ceph-mon[79813]: osdmap e51: 8 total, 8 up, 8 in 2023-06-23T01:16:08.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:07 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-23T01:16:08.264 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:07 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:08.264 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:07 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:08.264 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:07 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:16:08.264 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:07 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:16:08.264 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:07 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:16:08.264 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:07 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:08.264 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:07 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:16:08.265 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:07 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:16:08.265 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:07 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:08.632 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.c/config 2023-06-23T01:16:09.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:08 smithi157 ceph-mon[116945]: purged_snaps scrub starts 2023-06-23T01:16:09.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:08 smithi157 ceph-mon[116945]: purged_snaps scrub ok 2023-06-23T01:16:09.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:08 smithi157 ceph-mon[116945]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-06-23T01:16:09.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:08 smithi157 ceph-mon[116945]: Detected new or changed devices on smithi134 2023-06-23T01:16:09.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:08 smithi157 ceph-mon[116945]: Adjusting osd_memory_target on smithi134 to 5724M 2023-06-23T01:16:09.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:08 smithi157 ceph-mon[116945]: osdmap e52: 8 total, 8 up, 8 in 2023-06-23T01:16:09.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:08 smithi120 ceph-mon[79813]: purged_snaps scrub starts 2023-06-23T01:16:09.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:08 smithi120 ceph-mon[79813]: purged_snaps scrub ok 2023-06-23T01:16:09.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:08 smithi120 ceph-mon[79813]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-06-23T01:16:09.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:08 smithi120 ceph-mon[79813]: Detected new or changed devices on smithi134 2023-06-23T01:16:09.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:08 smithi120 ceph-mon[79813]: Adjusting osd_memory_target on smithi134 to 5724M 2023-06-23T01:16:09.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:08 smithi120 ceph-mon[79813]: osdmap e52: 8 total, 8 up, 8 in 2023-06-23T01:16:09.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:08 smithi134 ceph-mon[116146]: purged_snaps scrub starts 2023-06-23T01:16:09.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:08 smithi134 ceph-mon[116146]: purged_snaps scrub ok 2023-06-23T01:16:09.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:08 smithi134 ceph-mon[116146]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-06-23T01:16:09.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:08 smithi134 ceph-mon[116146]: Detected new or changed devices on smithi134 2023-06-23T01:16:09.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:08 smithi134 ceph-mon[116146]: Adjusting osd_memory_target on smithi134 to 5724M 2023-06-23T01:16:09.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:08 smithi134 ceph-mon[116146]: osdmap e52: 8 total, 8 up, 8 in 2023-06-23T01:16:10.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.002+0000 7effc3a95700 1 -- 172.21.15.157:0/2053351956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effbc0fe820 msgr2=0x7effbc1050e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:16:10.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.002+0000 7effc3a95700 1 --2- 172.21.15.157:0/2053351956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effbc0fe820 0x7effbc1050e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7effb40049c0 tx=0x7effb4009e30 comp rx=0 tx=0).stop 2023-06-23T01:16:10.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.002+0000 7effc3a95700 1 -- 172.21.15.157:0/2053351956 shutdown_connections 2023-06-23T01:16:10.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.002+0000 7effc3a95700 1 --2- 172.21.15.157:0/2053351956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effbc0fe820 0x7effbc1050e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:16:10.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.002+0000 7effc3a95700 1 --2- 172.21.15.157:0/2053351956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effbc0fcc80 0x7effbc0fd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:16:10.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.002+0000 7effc3a95700 1 --2- 172.21.15.157:0/2053351956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effbc0fde90 0x7effbc0fe2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:16:10.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.002+0000 7effc3a95700 1 -- 172.21.15.157:0/2053351956 >> 172.21.15.157:0/2053351956 conn(0x7effbc0f83a0 msgr2=0x7effbc0fa810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:16:10.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.002+0000 7effc3a95700 1 -- 172.21.15.157:0/2053351956 shutdown_connections 2023-06-23T01:16:10.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.002+0000 7effc3a95700 1 -- 172.21.15.157:0/2053351956 wait complete. 2023-06-23T01:16:10.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.003+0000 7effc3a95700 1 Processor -- start 2023-06-23T01:16:10.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.003+0000 7effc3a95700 1 -- start start 2023-06-23T01:16:10.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.003+0000 7effc3a95700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effbc0fcc80 0x7effbc10d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:16:10.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.003+0000 7effc3a95700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effbc0fde90 0x7effbc10dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:16:10.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.003+0000 7effc3a95700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effbc0fe820 0x7effbc114ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:16:10.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.003+0000 7effc3a95700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effbc1152b0 con 0x7effbc0fde90 2023-06-23T01:16:10.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.003+0000 7effc3a95700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effbc115420 con 0x7effbc0fe820 2023-06-23T01:16:10.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.003+0000 7effc3a95700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effbc1155e0 con 0x7effbc0fcc80 2023-06-23T01:16:10.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.003+0000 7effc1831700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effbc0fcc80 0x7effbc10d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:16:10.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.003+0000 7effc1831700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effbc0fcc80 0x7effbc10d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39174/0 (socket says 172.21.15.157:39174) 2023-06-23T01:16:10.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.003+0000 7effc1831700 1 -- 172.21.15.157:0/2059655561 learned_addr learned my addr 172.21.15.157:0/2059655561 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-06-23T01:16:10.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.003+0000 7effc1030700 1 --2- 172.21.15.157:0/2059655561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effbc0fde90 0x7effbc10dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:16:10.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.003+0000 7effc1831700 1 -- 172.21.15.157:0/2059655561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effbc0fe820 msgr2=0x7effbc114ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:16:10.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.003+0000 7effc1831700 1 --2- 172.21.15.157:0/2059655561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effbc0fe820 0x7effbc114ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:16:10.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.003+0000 7effc1831700 1 -- 172.21.15.157:0/2059655561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effbc0fde90 msgr2=0x7effbc10dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:16:10.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.003+0000 7effc1831700 1 --2- 172.21.15.157:0/2059655561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effbc0fde90 0x7effbc10dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:16:10.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.003+0000 7effc1831700 1 -- 172.21.15.157:0/2059655561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb4005040 con 0x7effbc0fcc80 2023-06-23T01:16:10.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.003+0000 7effc1030700 1 --2- 172.21.15.157:0/2059655561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effbc0fde90 0x7effbc10dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T01:16:10.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.003+0000 7effc1831700 1 --2- 172.21.15.157:0/2059655561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effbc0fcc80 0x7effbc10d6d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7effb800bb60 tx=0x7effb8009810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:16:10.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.004+0000 7effb27fc700 1 -- 172.21.15.157:0/2059655561 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb800c940 con 0x7effbc0fcc80 2023-06-23T01:16:10.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.004+0000 7effb27fc700 1 -- 172.21.15.157:0/2059655561 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effb8014410 con 0x7effbc0fcc80 2023-06-23T01:16:10.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.004+0000 7effc3a95700 1 -- 172.21.15.157:0/2059655561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effbc1158c0 con 0x7effbc0fcc80 2023-06-23T01:16:10.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.004+0000 7effb27fc700 1 -- 172.21.15.157:0/2059655561 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb8007640 con 0x7effbc0fcc80 2023-06-23T01:16:10.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.004+0000 7effc3a95700 1 -- 172.21.15.157:0/2059655561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effbc115db0 con 0x7effbc0fcc80 2023-06-23T01:16:10.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.004+0000 7effc3a95700 1 -- 172.21.15.157:0/2059655561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effbc1084a0 con 0x7effbc0fcc80 2023-06-23T01:16:10.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.006+0000 7effb27fc700 1 -- 172.21.15.157:0/2059655561 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7effb8014580 con 0x7effbc0fcc80 2023-06-23T01:16:10.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.006+0000 7effb27fc700 1 --2- 172.21.15.157:0/2059655561 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7effa809b680 0x7effa809db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:16:10.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.006+0000 7effb27fc700 1 -- 172.21.15.157:0/2059655561 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5393+0+0 (secure 0 0 0) 0x7effb80c48d0 con 0x7effbc0fcc80 2023-06-23T01:16:10.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.006+0000 7effc1030700 1 --2- 172.21.15.157:0/2059655561 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7effa809b680 0x7effa809db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:16:10.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.007+0000 7effc1030700 1 --2- 172.21.15.157:0/2059655561 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7effa809b680 0x7effa809db30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7effac007fd0 tx=0x7effac009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:16:10.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.009+0000 7effb27fc700 1 -- 172.21.15.157:0/2059655561 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7effb8090c90 con 0x7effbc0fcc80 2023-06-23T01:16:10.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:10.201+0000 7effc3a95700 1 -- 172.21.15.157:0/2059655561 --> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7effbc05de70 con 0x7effa809b680 2023-06-23T01:16:10.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:09 smithi120 ceph-mon[79813]: osdmap e53: 8 total, 8 up, 8 in 2023-06-23T01:16:10.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:09 smithi157 ceph-mon[116945]: osdmap e53: 8 total, 8 up, 8 in 2023-06-23T01:16:10.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:09 smithi134 ceph-mon[116146]: osdmap e53: 8 total, 8 up, 8 in 2023-06-23T01:16:11.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:10 smithi120 ceph-mon[79813]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-06-23T01:16:11.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:10 smithi120 ceph-mon[79813]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-06-23T01:16:11.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:10 smithi120 ceph-mon[79813]: osdmap e54: 8 total, 8 up, 8 in 2023-06-23T01:16:11.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:10 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T01:16:11.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:10 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T01:16:11.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:10 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:16:11.348 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:10 smithi157 ceph-mon[116945]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-06-23T01:16:11.349 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:10 smithi157 ceph-mon[116945]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-06-23T01:16:11.349 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:10 smithi157 ceph-mon[116945]: osdmap e54: 8 total, 8 up, 8 in 2023-06-23T01:16:11.349 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:10 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T01:16:11.349 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:10 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T01:16:11.349 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:10 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:16:11.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:10 smithi134 ceph-mon[116146]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-06-23T01:16:11.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:10 smithi134 ceph-mon[116146]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-06-23T01:16:11.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:10 smithi134 ceph-mon[116146]: osdmap e54: 8 total, 8 up, 8 in 2023-06-23T01:16:11.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:10 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T01:16:11.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:10 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T01:16:11.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:10 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:16:12.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:11 smithi157 ceph-mon[116945]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:16:12.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:11 smithi157 ceph-mon[116945]: osdmap e55: 8 total, 8 up, 8 in 2023-06-23T01:16:12.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:11 smithi120 ceph-mon[79813]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:16:12.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:11 smithi120 ceph-mon[79813]: osdmap e55: 8 total, 8 up, 8 in 2023-06-23T01:16:12.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:11 smithi134 ceph-mon[116146]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:16:12.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:11 smithi134 ceph-mon[116146]: osdmap e55: 8 total, 8 up, 8 in 2023-06-23T01:16:13.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:12 smithi120 ceph-mon[79813]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-06-23T01:16:13.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:12 smithi120 ceph-mon[79813]: from='client.? 172.21.15.157:0/105581849' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4aac47e-d7f9-4feb-ada7-d8103fdca246"}]: dispatch 2023-06-23T01:16:13.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:12 smithi120 ceph-mon[79813]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4aac47e-d7f9-4feb-ada7-d8103fdca246"}]: dispatch 2023-06-23T01:16:13.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:12 smithi120 ceph-mon[79813]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4aac47e-d7f9-4feb-ada7-d8103fdca246"}]': finished 2023-06-23T01:16:13.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:12 smithi120 ceph-mon[79813]: osdmap e56: 9 total, 8 up, 9 in 2023-06-23T01:16:13.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:12 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-23T01:16:13.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:12 smithi157 ceph-mon[116945]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-06-23T01:16:13.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:12 smithi157 ceph-mon[116945]: from='client.? 172.21.15.157:0/105581849' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4aac47e-d7f9-4feb-ada7-d8103fdca246"}]: dispatch 2023-06-23T01:16:13.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:12 smithi157 ceph-mon[116945]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4aac47e-d7f9-4feb-ada7-d8103fdca246"}]: dispatch 2023-06-23T01:16:13.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:12 smithi157 ceph-mon[116945]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4aac47e-d7f9-4feb-ada7-d8103fdca246"}]': finished 2023-06-23T01:16:13.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:12 smithi157 ceph-mon[116945]: osdmap e56: 9 total, 8 up, 9 in 2023-06-23T01:16:13.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:12 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-23T01:16:13.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:12 smithi134 ceph-mon[116146]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-06-23T01:16:13.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:12 smithi134 ceph-mon[116146]: from='client.? 172.21.15.157:0/105581849' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4aac47e-d7f9-4feb-ada7-d8103fdca246"}]: dispatch 2023-06-23T01:16:13.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:12 smithi134 ceph-mon[116146]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4aac47e-d7f9-4feb-ada7-d8103fdca246"}]: dispatch 2023-06-23T01:16:13.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:12 smithi134 ceph-mon[116146]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4aac47e-d7f9-4feb-ada7-d8103fdca246"}]': finished 2023-06-23T01:16:13.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:12 smithi134 ceph-mon[116146]: osdmap e56: 9 total, 8 up, 9 in 2023-06-23T01:16:13.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:12 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-23T01:16:14.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:13 smithi120 ceph-mon[79813]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-06-23T01:16:14.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:13 smithi120 ceph-mon[79813]: from='client.? 172.21.15.157:0/653612294' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T01:16:14.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:13 smithi157 ceph-mon[116945]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-06-23T01:16:14.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:13 smithi157 ceph-mon[116945]: from='client.? 172.21.15.157:0/653612294' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T01:16:14.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:13 smithi134 ceph-mon[116146]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-06-23T01:16:14.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:13 smithi134 ceph-mon[116146]: from='client.? 172.21.15.157:0/653612294' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T01:16:15.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:14 smithi120 ceph-mon[79813]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2023-06-23T01:16:15.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:14 smithi120 ceph-mon[79813]: Cluster is now healthy 2023-06-23T01:16:15.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:14 smithi157 ceph-mon[116945]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2023-06-23T01:16:15.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:14 smithi157 ceph-mon[116945]: Cluster is now healthy 2023-06-23T01:16:15.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:14 smithi134 ceph-mon[116146]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2023-06-23T01:16:15.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:14 smithi134 ceph-mon[116146]: Cluster is now healthy 2023-06-23T01:16:16.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:15 smithi120 ceph-mon[79813]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-06-23T01:16:16.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:15 smithi157 ceph-mon[116945]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-06-23T01:16:16.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:15 smithi134 ceph-mon[116146]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-06-23T01:16:18.971 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:18 smithi157 ceph-mon[116945]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-06-23T01:16:19.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:18 smithi120 ceph-mon[79813]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-06-23T01:16:19.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:18 smithi134 ceph-mon[116146]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-06-23T01:16:21.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:20 smithi120 ceph-mon[79813]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-06-23T01:16:21.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:20 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-06-23T01:16:21.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:20 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:16:21.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:20 smithi157 ceph-mon[116945]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-06-23T01:16:21.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:20 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-06-23T01:16:21.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:20 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:16:21.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:20 smithi134 ceph-mon[116146]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-06-23T01:16:21.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:20 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-06-23T01:16:21.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:20 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:16:22.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:21 smithi120 ceph-mon[79813]: Deploying daemon osd.8 on smithi157 2023-06-23T01:16:22.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:21 smithi157 ceph-mon[116945]: Deploying daemon osd.8 on smithi157 2023-06-23T01:16:22.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:21 smithi134 ceph-mon[116146]: Deploying daemon osd.8 on smithi157 2023-06-23T01:16:23.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:22 smithi120 ceph-mon[79813]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-06-23T01:16:23.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:22 smithi157 ceph-mon[116945]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-06-23T01:16:23.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:22 smithi134 ceph-mon[116146]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-06-23T01:16:25.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:24 smithi120 ceph-mon[79813]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-06-23T01:16:25.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:24 smithi157 ceph-mon[116945]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-06-23T01:16:25.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:24 smithi134 ceph-mon[116146]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-06-23T01:16:26.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:26 smithi157 ceph-mon[116945]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-06-23T01:16:26.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:26 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:26.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:26 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:26.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:26 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:16:26.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:26 smithi134 ceph-mon[116146]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-06-23T01:16:26.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:26 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:26.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:26 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:26.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:26 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:16:26.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:26 smithi120 ceph-mon[79813]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-06-23T01:16:26.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:26 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:26.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:26 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:26.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:26 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:16:28.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:28.619+0000 7effb27fc700 1 -- 172.21.15.157:0/2059655561 <== mgr.14152 v2:172.21.15.120:6800/1770912857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7effbc05de70 con 0x7effa809b680 2023-06-23T01:16:28.624 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 8 on host 'smithi157' 2023-06-23T01:16:28.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:28.621+0000 7effc3a95700 1 -- 172.21.15.157:0/2059655561 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7effa809b680 msgr2=0x7effa809db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:16:28.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:28.621+0000 7effc3a95700 1 --2- 172.21.15.157:0/2059655561 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7effa809b680 0x7effa809db30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7effac007fd0 tx=0x7effac009040 comp rx=0 tx=0).stop 2023-06-23T01:16:28.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:28.621+0000 7effc3a95700 1 -- 172.21.15.157:0/2059655561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effbc0fcc80 msgr2=0x7effbc10d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:16:28.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:28.621+0000 7effc3a95700 1 --2- 172.21.15.157:0/2059655561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effbc0fcc80 0x7effbc10d6d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7effb800bb60 tx=0x7effb8009810 comp rx=0 tx=0).stop 2023-06-23T01:16:28.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:28.621+0000 7effc3a95700 1 -- 172.21.15.157:0/2059655561 shutdown_connections 2023-06-23T01:16:28.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:28.621+0000 7effc3a95700 1 --2- 172.21.15.157:0/2059655561 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7effa809b680 0x7effa809db30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:16:28.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:28.621+0000 7effc3a95700 1 --2- 172.21.15.157:0/2059655561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effbc0fe820 0x7effbc114ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:16:28.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:28.621+0000 7effc3a95700 1 --2- 172.21.15.157:0/2059655561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effbc0fcc80 0x7effbc10d6d0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:16:28.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:28.621+0000 7effc3a95700 1 --2- 172.21.15.157:0/2059655561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effbc0fde90 0x7effbc10dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:16:28.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:28.621+0000 7effc3a95700 1 -- 172.21.15.157:0/2059655561 >> 172.21.15.157:0/2059655561 conn(0x7effbc0f83a0 msgr2=0x7effbc0fa410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:16:28.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:28.622+0000 7effc3a95700 1 -- 172.21.15.157:0/2059655561 shutdown_connections 2023-06-23T01:16:28.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:16:28.622+0000 7effc3a95700 1 -- 172.21.15.157:0/2059655561 wait complete. 2023-06-23T01:16:28.876 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:28 smithi157 ceph-mon[116945]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-06-23T01:16:28.876 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:28 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:28.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:28 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:28.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:28 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:16:28.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:28 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:16:28.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:28 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:28.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:28 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:28.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:28 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:28.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:28 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:16:29.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:28 smithi120 ceph-mon[79813]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-06-23T01:16:29.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:28 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:29.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:28 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:29.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:28 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:16:29.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:28 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:16:29.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:28 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:29.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:28 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:29.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:28 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:29.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:28 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:16:29.183 DEBUG:teuthology.orchestra.run.smithi157:osd.8> sudo journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@osd.8.service 2023-06-23T01:16:29.187 INFO:tasks.cephadm:Deploying osd.9 on smithi120 with /dev/vg_nvme/lv_1... 2023-06-23T01:16:29.187 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-06-23T01:16:29.193 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:28 smithi134 ceph-mon[116146]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-06-23T01:16:29.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:29.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:29.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:16:29.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:16:29.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:29.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:29.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:29.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:28 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:16:29.214 INFO:journalctl@ceph.osd.8.smithi157.stdout:-- Logs begin at Fri 2023-06-23 00:49:55 UTC. -- 2023-06-23T01:16:29.876 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:29 smithi157 ceph-mon[116945]: from='osd.8 [v2:172.21.15.157:6816/1124549599,v1:172.21.15.157:6817/1124549599]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-06-23T01:16:29.876 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:29 smithi157 ceph-mon[116945]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-06-23T01:16:29.952 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T01:16:30.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:29 smithi120 ceph-mon[79813]: from='osd.8 [v2:172.21.15.157:6816/1124549599,v1:172.21.15.157:6817/1124549599]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-06-23T01:16:30.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:29 smithi120 ceph-mon[79813]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-06-23T01:16:30.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:29 smithi134 ceph-mon[116146]: from='osd.8 [v2:172.21.15.157:6816/1124549599,v1:172.21.15.157:6817/1124549599]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-06-23T01:16:30.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:29 smithi134 ceph-mon[116146]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-06-23T01:16:31.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:30 smithi120 ceph-mon[79813]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-06-23T01:16:31.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:30 smithi120 ceph-mon[79813]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-06-23T01:16:31.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:30 smithi120 ceph-mon[79813]: osdmap e57: 9 total, 8 up, 9 in 2023-06-23T01:16:31.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:30 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-23T01:16:31.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:30 smithi120 ceph-mon[79813]: from='osd.8 [v2:172.21.15.157:6816/1124549599,v1:172.21.15.157:6817/1124549599]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-06-23T01:16:31.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:30 smithi120 ceph-mon[79813]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-06-23T01:16:31.123 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:30 smithi157 ceph-mon[116945]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-06-23T01:16:31.124 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:30 smithi157 ceph-mon[116945]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-06-23T01:16:31.124 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:30 smithi157 ceph-mon[116945]: osdmap e57: 9 total, 8 up, 9 in 2023-06-23T01:16:31.125 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:30 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-23T01:16:31.125 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:30 smithi157 ceph-mon[116945]: from='osd.8 [v2:172.21.15.157:6816/1124549599,v1:172.21.15.157:6817/1124549599]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-06-23T01:16:31.125 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:30 smithi157 ceph-mon[116945]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-06-23T01:16:31.126 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:16:30 smithi157 conmon[138256]: 2023-06-23T01:16:30.739+0000 7fd2e7e47700 -1 osd.8 0 waiting for initial osdmap 2023-06-23T01:16:31.126 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:16:30 smithi157 conmon[138256]: 2023-06-23T01:16:30.753+0000 7fd2e4c46700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T01:16:31.193 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:30 smithi134 ceph-mon[116146]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-06-23T01:16:31.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:30 smithi134 ceph-mon[116146]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-06-23T01:16:31.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:30 smithi134 ceph-mon[116146]: osdmap e57: 9 total, 8 up, 9 in 2023-06-23T01:16:31.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:30 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-23T01:16:31.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:30 smithi134 ceph-mon[116146]: from='osd.8 [v2:172.21.15.157:6816/1124549599,v1:172.21.15.157:6817/1124549599]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-06-23T01:16:31.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:30 smithi134 ceph-mon[116146]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-06-23T01:16:31.916 INFO:teuthology.orchestra.run.smithi120.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-06-23T01:16:31.917 INFO:teuthology.orchestra.run.smithi120.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-06-23T01:16:31.917 INFO:teuthology.orchestra.run.smithi120.stderr: stderr: 10+0 records in 2023-06-23T01:16:31.917 INFO:teuthology.orchestra.run.smithi120.stderr:10+0 records out 2023-06-23T01:16:31.917 INFO:teuthology.orchestra.run.smithi120.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0461676 s, 227 MB/s 2023-06-23T01:16:31.917 INFO:teuthology.orchestra.run.smithi120.stderr:--> Zapping successful for: 2023-06-23T01:16:32.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:31 smithi120 ceph-mon[79813]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2023-06-23T01:16:32.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:31 smithi120 ceph-mon[79813]: osdmap e58: 9 total, 8 up, 9 in 2023-06-23T01:16:32.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:31 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-23T01:16:32.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:31 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-23T01:16:32.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:31 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:32.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:31 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:32.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:31 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:16:32.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:31 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:16:32.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:31 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:16:32.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:31 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:32.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:31 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:16:32.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:31 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:16:32.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:31 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:32.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:31 smithi157 ceph-mon[116945]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2023-06-23T01:16:32.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:31 smithi157 ceph-mon[116945]: osdmap e58: 9 total, 8 up, 9 in 2023-06-23T01:16:32.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:31 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-23T01:16:32.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:31 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-23T01:16:32.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:31 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:32.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:31 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:32.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:31 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:16:32.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:31 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:16:32.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:31 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:16:32.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:31 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:32.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:31 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:16:32.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:31 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:16:32.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:31 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:32.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:31 smithi134 ceph-mon[116146]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2023-06-23T01:16:32.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:31 smithi134 ceph-mon[116146]: osdmap e58: 9 total, 8 up, 9 in 2023-06-23T01:16:32.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:31 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-23T01:16:32.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:31 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-23T01:16:32.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:31 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:32.196 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:31 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:32.196 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:31 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:16:32.196 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:31 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:16:32.196 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:31 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:16:32.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:31 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:32.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:31 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:16:32.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:31 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:16:32.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:31 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:32.312 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph orch daemon add osd smithi120:vg_nvme/lv_1 2023-06-23T01:16:33.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:32 smithi120 ceph-mon[79813]: purged_snaps scrub starts 2023-06-23T01:16:33.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:32 smithi120 ceph-mon[79813]: purged_snaps scrub ok 2023-06-23T01:16:33.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:32 smithi120 ceph-mon[79813]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-06-23T01:16:33.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:32 smithi120 ceph-mon[79813]: Detected new or changed devices on smithi157 2023-06-23T01:16:33.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:32 smithi120 ceph-mon[79813]: Adjusting osd_memory_target on smithi157 to 5724M 2023-06-23T01:16:33.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:32 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-23T01:16:33.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:32 smithi120 ceph-mon[79813]: osd.8 [v2:172.21.15.157:6816/1124549599,v1:172.21.15.157:6817/1124549599] boot 2023-06-23T01:16:33.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:32 smithi120 ceph-mon[79813]: osdmap e59: 9 total, 9 up, 9 in 2023-06-23T01:16:33.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:32 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-23T01:16:33.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:32 smithi120 ceph-mon[79813]: osdmap e60: 9 total, 9 up, 9 in 2023-06-23T01:16:33.073 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T01:16:33.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:32 smithi157 ceph-mon[116945]: purged_snaps scrub starts 2023-06-23T01:16:33.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:32 smithi157 ceph-mon[116945]: purged_snaps scrub ok 2023-06-23T01:16:33.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:32 smithi157 ceph-mon[116945]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-06-23T01:16:33.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:32 smithi157 ceph-mon[116945]: Detected new or changed devices on smithi157 2023-06-23T01:16:33.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:32 smithi157 ceph-mon[116945]: Adjusting osd_memory_target on smithi157 to 5724M 2023-06-23T01:16:33.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:32 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-23T01:16:33.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:32 smithi157 ceph-mon[116945]: osd.8 [v2:172.21.15.157:6816/1124549599,v1:172.21.15.157:6817/1124549599] boot 2023-06-23T01:16:33.129 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:32 smithi157 ceph-mon[116945]: osdmap e59: 9 total, 9 up, 9 in 2023-06-23T01:16:33.129 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:32 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-23T01:16:33.129 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:32 smithi157 ceph-mon[116945]: osdmap e60: 9 total, 9 up, 9 in 2023-06-23T01:16:33.193 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:32 smithi134 ceph-mon[116146]: purged_snaps scrub starts 2023-06-23T01:16:33.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:32 smithi134 ceph-mon[116146]: purged_snaps scrub ok 2023-06-23T01:16:33.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:32 smithi134 ceph-mon[116146]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-06-23T01:16:33.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:32 smithi134 ceph-mon[116146]: Detected new or changed devices on smithi157 2023-06-23T01:16:33.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:32 smithi134 ceph-mon[116146]: Adjusting osd_memory_target on smithi157 to 5724M 2023-06-23T01:16:33.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:32 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-23T01:16:33.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:32 smithi134 ceph-mon[116146]: osd.8 [v2:172.21.15.157:6816/1124549599,v1:172.21.15.157:6817/1124549599] boot 2023-06-23T01:16:33.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:32 smithi134 ceph-mon[116146]: osdmap e59: 9 total, 9 up, 9 in 2023-06-23T01:16:33.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:32 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-23T01:16:33.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:32 smithi134 ceph-mon[116146]: osdmap e60: 9 total, 9 up, 9 in 2023-06-23T01:16:34.247 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:33 smithi120 ceph-mon[79813]: osdmap e61: 9 total, 9 up, 9 in 2023-06-23T01:16:34.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:33 smithi157 ceph-mon[116945]: osdmap e61: 9 total, 9 up, 9 in 2023-06-23T01:16:34.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:33 smithi134 ceph-mon[116146]: osdmap e61: 9 total, 9 up, 9 in 2023-06-23T01:16:34.483 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.479+0000 7fa81d3c3700 1 -- 172.21.15.120:0/1856181538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8180fde90 msgr2=0x7fa8180fe2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:16:34.483 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.479+0000 7fa81d3c3700 1 --2- 172.21.15.120:0/1856181538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8180fde90 0x7fa8180fe2e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa80c0049c0 tx=0x7fa80c009e30 comp rx=0 tx=0).stop 2023-06-23T01:16:34.484 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.479+0000 7fa81d3c3700 1 -- 172.21.15.120:0/1856181538 shutdown_connections 2023-06-23T01:16:34.484 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.479+0000 7fa81d3c3700 1 --2- 172.21.15.120:0/1856181538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8180fe820 0x7fa8181050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:16:34.484 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.479+0000 7fa81d3c3700 1 --2- 172.21.15.120:0/1856181538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8180fcc80 0x7fa8180fd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:16:34.484 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.479+0000 7fa81d3c3700 1 --2- 172.21.15.120:0/1856181538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8180fde90 0x7fa8180fe2e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:16:34.484 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.479+0000 7fa81d3c3700 1 -- 172.21.15.120:0/1856181538 >> 172.21.15.120:0/1856181538 conn(0x7fa8180f83a0 msgr2=0x7fa8180fa810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:16:34.484 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.479+0000 7fa81d3c3700 1 -- 172.21.15.120:0/1856181538 shutdown_connections 2023-06-23T01:16:34.484 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.479+0000 7fa81d3c3700 1 -- 172.21.15.120:0/1856181538 wait complete. 2023-06-23T01:16:34.486 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.480+0000 7fa81d3c3700 1 Processor -- start 2023-06-23T01:16:34.486 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.480+0000 7fa81d3c3700 1 -- start start 2023-06-23T01:16:34.486 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.480+0000 7fa81d3c3700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8180fcc80 0x7fa81810faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:16:34.486 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.480+0000 7fa81d3c3700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8180fde90 0x7fa818110030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:16:34.486 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.480+0000 7fa81d3c3700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8180fe820 0x7fa8181170c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:16:34.487 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.480+0000 7fa81d3c3700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8181176d0 con 0x7fa8180fcc80 2023-06-23T01:16:34.487 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.480+0000 7fa81d3c3700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa818117840 con 0x7fa8180fe820 2023-06-23T01:16:34.487 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.480+0000 7fa81d3c3700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa818117a00 con 0x7fa8180fde90 2023-06-23T01:16:34.487 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.480+0000 7fa816ffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8180fcc80 0x7fa81810faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:16:34.487 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.480+0000 7fa816ffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8180fcc80 0x7fa81810faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44304/0 (socket says 172.21.15.120:44304) 2023-06-23T01:16:34.487 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.480+0000 7fa816ffd700 1 -- 172.21.15.120:0/1634887401 learned_addr learned my addr 172.21.15.120:0/1634887401 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:16:34.487 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.480+0000 7fa8177fe700 1 --2- 172.21.15.120:0/1634887401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8180fe820 0x7fa8181170c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:16:34.488 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.480+0000 7fa8167fc700 1 --2- 172.21.15.120:0/1634887401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8180fde90 0x7fa818110030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:16:34.488 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.481+0000 7fa816ffd700 1 -- 172.21.15.120:0/1634887401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8180fde90 msgr2=0x7fa818110030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:16:34.488 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.481+0000 7fa816ffd700 1 --2- 172.21.15.120:0/1634887401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8180fde90 0x7fa818110030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:16:34.488 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.481+0000 7fa816ffd700 1 -- 172.21.15.120:0/1634887401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8180fe820 msgr2=0x7fa8181170c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:16:34.488 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.481+0000 7fa816ffd700 1 --2- 172.21.15.120:0/1634887401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8180fe820 0x7fa8181170c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:16:34.488 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.481+0000 7fa816ffd700 1 -- 172.21.15.120:0/1634887401 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa80c005040 con 0x7fa8180fcc80 2023-06-23T01:16:34.488 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.481+0000 7fa816ffd700 1 --2- 172.21.15.120:0/1634887401 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8180fcc80 0x7fa81810faf0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa80800ebf0 tx=0x7fa80800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:16:34.489 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.481+0000 7fa7ff7fe700 1 -- 172.21.15.120:0/1634887401 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa808007cc0 con 0x7fa8180fcc80 2023-06-23T01:16:34.489 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.481+0000 7fa81d3c3700 1 -- 172.21.15.120:0/1634887401 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa818117c80 con 0x7fa8180fcc80 2023-06-23T01:16:34.489 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.481+0000 7fa7ff7fe700 1 -- 172.21.15.120:0/1634887401 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa80800f040 con 0x7fa8180fcc80 2023-06-23T01:16:34.489 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.481+0000 7fa7ff7fe700 1 -- 172.21.15.120:0/1634887401 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80800c8f0 con 0x7fa8180fcc80 2023-06-23T01:16:34.489 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.481+0000 7fa81d3c3700 1 -- 172.21.15.120:0/1634887401 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa818118170 con 0x7fa8180fcc80 2023-06-23T01:16:34.490 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.482+0000 7fa81d3c3700 1 -- 172.21.15.120:0/1634887401 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa818108450 con 0x7fa8180fcc80 2023-06-23T01:16:34.490 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.484+0000 7fa7ff7fe700 1 -- 172.21.15.120:0/1634887401 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7fa80800cd20 con 0x7fa8180fcc80 2023-06-23T01:16:34.490 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.485+0000 7fa7ff7fe700 1 --2- 172.21.15.120:0/1634887401 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fa80009b680 0x7fa80009db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:16:34.491 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.485+0000 7fa7ff7fe700 1 -- 172.21.15.120:0/1634887401 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fa8080c3ee0 con 0x7fa8180fcc80 2023-06-23T01:16:34.491 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.485+0000 7fa8167fc700 1 --2- 172.21.15.120:0/1634887401 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fa80009b680 0x7fa80009db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:16:34.491 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.486+0000 7fa8167fc700 1 --2- 172.21.15.120:0/1634887401 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fa80009b680 0x7fa80009db30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa80c007f50 tx=0x7fa80c009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:16:34.494 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.489+0000 7fa7ff7fe700 1 -- 172.21.15.120:0/1634887401 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fa80804da00 con 0x7fa8180fcc80 2023-06-23T01:16:34.693 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:34.689+0000 7fa81d3c3700 1 -- 172.21.15.120:0/1634887401 --> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa818104e40 con 0x7fa80009b680 2023-06-23T01:16:35.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:34 smithi120 ceph-mon[79813]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 54 MiB used, 805 GiB / 805 GiB avail 2023-06-23T01:16:35.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:34 smithi120 ceph-mon[79813]: osdmap e62: 9 total, 9 up, 9 in 2023-06-23T01:16:35.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:34 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T01:16:35.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:34 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T01:16:35.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:34 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:16:35.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:34 smithi157 ceph-mon[116945]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 54 MiB used, 805 GiB / 805 GiB avail 2023-06-23T01:16:35.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:34 smithi157 ceph-mon[116945]: osdmap e62: 9 total, 9 up, 9 in 2023-06-23T01:16:35.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:34 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T01:16:35.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:34 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T01:16:35.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:34 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:16:35.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:34 smithi134 ceph-mon[116146]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 54 MiB used, 805 GiB / 805 GiB avail 2023-06-23T01:16:35.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:34 smithi134 ceph-mon[116146]: osdmap e62: 9 total, 9 up, 9 in 2023-06-23T01:16:35.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:34 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T01:16:35.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:34 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T01:16:35.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:34 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:16:36.190 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:35 smithi120 ceph-mon[79813]: from='client.14331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:16:36.191 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:35 smithi120 ceph-mon[79813]: osdmap e63: 9 total, 9 up, 9 in 2023-06-23T01:16:36.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:35 smithi157 ceph-mon[116945]: from='client.14331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:16:36.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:35 smithi157 ceph-mon[116945]: osdmap e63: 9 total, 9 up, 9 in 2023-06-23T01:16:36.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:35 smithi134 ceph-mon[116146]: from='client.14331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:16:36.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:35 smithi134 ceph-mon[116146]: osdmap e63: 9 total, 9 up, 9 in 2023-06-23T01:16:37.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:36 smithi120 ceph-mon[79813]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 54 MiB used, 805 GiB / 805 GiB avail 2023-06-23T01:16:37.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:36 smithi157 ceph-mon[116945]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 54 MiB used, 805 GiB / 805 GiB avail 2023-06-23T01:16:37.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:36 smithi134 ceph-mon[116146]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 54 MiB used, 805 GiB / 805 GiB avail 2023-06-23T01:16:38.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:37 smithi120 ceph-mon[79813]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 54 MiB used, 805 GiB / 805 GiB avail 2023-06-23T01:16:38.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:37 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/793577304' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48b988ba-e866-47d1-aca9-09f6ad815bd8"}]: dispatch 2023-06-23T01:16:38.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:37 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/793577304' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "48b988ba-e866-47d1-aca9-09f6ad815bd8"}]': finished 2023-06-23T01:16:38.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:37 smithi120 ceph-mon[79813]: osdmap e64: 10 total, 9 up, 10 in 2023-06-23T01:16:38.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:37 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-23T01:16:38.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:37 smithi157 ceph-mon[116945]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 54 MiB used, 805 GiB / 805 GiB avail 2023-06-23T01:16:38.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:37 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/793577304' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48b988ba-e866-47d1-aca9-09f6ad815bd8"}]: dispatch 2023-06-23T01:16:38.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:37 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/793577304' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "48b988ba-e866-47d1-aca9-09f6ad815bd8"}]': finished 2023-06-23T01:16:38.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:37 smithi157 ceph-mon[116945]: osdmap e64: 10 total, 9 up, 10 in 2023-06-23T01:16:38.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:37 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-23T01:16:38.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:37 smithi134 ceph-mon[116146]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 54 MiB used, 805 GiB / 805 GiB avail 2023-06-23T01:16:38.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:37 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/793577304' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48b988ba-e866-47d1-aca9-09f6ad815bd8"}]: dispatch 2023-06-23T01:16:38.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:37 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/793577304' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "48b988ba-e866-47d1-aca9-09f6ad815bd8"}]': finished 2023-06-23T01:16:38.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:37 smithi134 ceph-mon[116146]: osdmap e64: 10 total, 9 up, 10 in 2023-06-23T01:16:38.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:37 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-23T01:16:39.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:38 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/4116658525' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T01:16:39.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:38 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/4116658525' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T01:16:39.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:38 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/4116658525' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T01:16:40.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:40 smithi120 ceph-mon[79813]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-06-23T01:16:40.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:40 smithi157 ceph-mon[116945]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-06-23T01:16:40.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:40 smithi134 ceph-mon[116146]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-06-23T01:16:43.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:42 smithi120 ceph-mon[79813]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2023-06-23T01:16:43.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:42 smithi157 ceph-mon[116945]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2023-06-23T01:16:43.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:42 smithi134 ceph-mon[116146]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2023-06-23T01:16:45.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:44 smithi120 ceph-mon[79813]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-06-23T01:16:45.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:44 smithi157 ceph-mon[116945]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-06-23T01:16:45.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:44 smithi134 ceph-mon[116146]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-06-23T01:16:46.242 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:45 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-06-23T01:16:46.243 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:45 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:16:46.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:45 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-06-23T01:16:46.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:45 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:16:46.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:45 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-06-23T01:16:46.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:45 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:16:47.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:46 smithi120 ceph-mon[79813]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-06-23T01:16:47.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:46 smithi120 ceph-mon[79813]: Deploying daemon osd.9 on smithi120 2023-06-23T01:16:47.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:46 smithi157 ceph-mon[116945]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-06-23T01:16:47.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:46 smithi157 ceph-mon[116945]: Deploying daemon osd.9 on smithi120 2023-06-23T01:16:47.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:46 smithi134 ceph-mon[116146]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-06-23T01:16:47.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:46 smithi134 ceph-mon[116146]: Deploying daemon osd.9 on smithi120 2023-06-23T01:16:49.256 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:48 smithi120 ceph-mon[79813]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-06-23T01:16:49.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:48 smithi157 ceph-mon[116945]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-06-23T01:16:49.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:48 smithi134 ceph-mon[116146]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-06-23T01:16:51.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:50 smithi120 ceph-mon[79813]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-06-23T01:16:51.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:50 smithi157 ceph-mon[116945]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-06-23T01:16:51.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:50 smithi134 ceph-mon[116146]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-06-23T01:16:52.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:52 smithi134 ceph-mon[116146]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-06-23T01:16:52.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:52 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:52.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:52 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:52.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:52 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:16:52.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:52 smithi120 ceph-mon[79813]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-06-23T01:16:52.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:52 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:52.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:52 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:52.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:52 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:16:52.626 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:52 smithi157 ceph-mon[116945]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-06-23T01:16:52.627 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:52 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:52.627 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:52 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:52.628 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:52 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:16:54.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:54 smithi134 ceph-mon[116146]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-06-23T01:16:54.505 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:54 smithi120 ceph-mon[79813]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-06-23T01:16:54.626 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:54 smithi157 ceph-mon[116945]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-06-23T01:16:54.898 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:54.895+0000 7fa7ff7fe700 1 -- 172.21.15.120:0/1634887401 <== mgr.14152 v2:172.21.15.120:6800/1770912857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa818104e40 con 0x7fa80009b680 2023-06-23T01:16:54.899 INFO:teuthology.orchestra.run.smithi120.stdout:Created osd(s) 9 on host 'smithi120' 2023-06-23T01:16:54.901 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:54.898+0000 7fa81d3c3700 1 -- 172.21.15.120:0/1634887401 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fa80009b680 msgr2=0x7fa80009db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:16:54.902 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:54.898+0000 7fa81d3c3700 1 --2- 172.21.15.120:0/1634887401 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fa80009b680 0x7fa80009db30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa80c007f50 tx=0x7fa80c009e30 comp rx=0 tx=0).stop 2023-06-23T01:16:54.902 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:54.898+0000 7fa81d3c3700 1 -- 172.21.15.120:0/1634887401 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8180fcc80 msgr2=0x7fa81810faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:16:54.902 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:54.898+0000 7fa81d3c3700 1 --2- 172.21.15.120:0/1634887401 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8180fcc80 0x7fa81810faf0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa80800ebf0 tx=0x7fa80800b040 comp rx=0 tx=0).stop 2023-06-23T01:16:54.902 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:54.898+0000 7fa81d3c3700 1 -- 172.21.15.120:0/1634887401 shutdown_connections 2023-06-23T01:16:54.902 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:54.898+0000 7fa81d3c3700 1 --2- 172.21.15.120:0/1634887401 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fa80009b680 0x7fa80009db30 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:16:54.902 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:54.898+0000 7fa81d3c3700 1 --2- 172.21.15.120:0/1634887401 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8180fcc80 0x7fa81810faf0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:16:54.902 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:54.898+0000 7fa81d3c3700 1 --2- 172.21.15.120:0/1634887401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8180fe820 0x7fa8181170c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:16:54.903 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:54.898+0000 7fa81d3c3700 1 --2- 172.21.15.120:0/1634887401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8180fde90 0x7fa818110030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:16:54.903 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:54.898+0000 7fa81d3c3700 1 -- 172.21.15.120:0/1634887401 >> 172.21.15.120:0/1634887401 conn(0x7fa8180f83a0 msgr2=0x7fa818103720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:16:54.903 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:54.899+0000 7fa81d3c3700 1 -- 172.21.15.120:0/1634887401 shutdown_connections 2023-06-23T01:16:54.903 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:16:54.899+0000 7fa81d3c3700 1 -- 172.21.15.120:0/1634887401 wait complete. 2023-06-23T01:16:55.353 DEBUG:teuthology.orchestra.run.smithi120:osd.9> sudo journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@osd.9.service 2023-06-23T01:16:55.358 INFO:tasks.cephadm:Deploying osd.10 on smithi134 with /dev/vg_nvme/lv_1... 2023-06-23T01:16:55.358 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-06-23T01:16:55.423 INFO:journalctl@ceph.osd.9.smithi120.stdout:-- Logs begin at Fri 2023-06-23 00:53:53 UTC. -- 2023-06-23T01:16:55.693 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:55 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:55.694 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:55 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:55.694 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:55 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:16:55.694 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:55 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:16:55.694 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:55 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:55.694 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:55 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:55.695 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:55 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:55.695 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:55 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:16:55.695 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:55 smithi134 ceph-mon[116146]: from='osd.9 [v2:172.21.15.120:6826/1402414981,v1:172.21.15.120:6827/1402414981]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-06-23T01:16:55.876 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:55 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:55.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:55 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:55.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:55 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:16:55.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:55 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:16:55.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:55 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:55.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:55 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:55.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:55 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:55.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:55 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:16:55.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:55 smithi157 ceph-mon[116945]: from='osd.9 [v2:172.21.15.120:6826/1402414981,v1:172.21.15.120:6827/1402414981]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-06-23T01:16:55.946 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.b/config 2023-06-23T01:16:56.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:55 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:56.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:55 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:56.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:55 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:16:56.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:55 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:16:56.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:55 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:56.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:55 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:56.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:55 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:56.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:55 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:16:56.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:55 smithi120 ceph-mon[79813]: from='osd.9 [v2:172.21.15.120:6826/1402414981,v1:172.21.15.120:6827/1402414981]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-06-23T01:16:57.140 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:56 smithi134 ceph-mon[116146]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-06-23T01:16:57.141 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:56 smithi134 ceph-mon[116146]: from='osd.9 [v2:172.21.15.120:6826/1402414981,v1:172.21.15.120:6827/1402414981]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-06-23T01:16:57.141 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:56 smithi134 ceph-mon[116146]: osdmap e65: 10 total, 9 up, 10 in 2023-06-23T01:16:57.141 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:56 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-23T01:16:57.141 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:56 smithi134 ceph-mon[116146]: from='osd.9 [v2:172.21.15.120:6826/1402414981,v1:172.21.15.120:6827/1402414981]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2023-06-23T01:16:57.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:56 smithi120 ceph-mon[79813]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-06-23T01:16:57.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:56 smithi120 ceph-mon[79813]: from='osd.9 [v2:172.21.15.120:6826/1402414981,v1:172.21.15.120:6827/1402414981]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-06-23T01:16:57.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:56 smithi120 ceph-mon[79813]: osdmap e65: 10 total, 9 up, 10 in 2023-06-23T01:16:57.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:56 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-23T01:16:57.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:56 smithi120 ceph-mon[79813]: from='osd.9 [v2:172.21.15.120:6826/1402414981,v1:172.21.15.120:6827/1402414981]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2023-06-23T01:16:57.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:16:56 smithi120 conmon[117204]: 2023-06-23T01:16:56.888+0000 7f748a8b8700 -1 osd.9 0 waiting for initial osdmap 2023-06-23T01:16:57.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:16:56 smithi120 conmon[117204]: 2023-06-23T01:16:56.908+0000 7f74836af700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T01:16:57.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:56 smithi157 ceph-mon[116945]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-06-23T01:16:57.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:56 smithi157 ceph-mon[116945]: from='osd.9 [v2:172.21.15.120:6826/1402414981,v1:172.21.15.120:6827/1402414981]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-06-23T01:16:57.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:56 smithi157 ceph-mon[116945]: osdmap e65: 10 total, 9 up, 10 in 2023-06-23T01:16:57.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:56 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-23T01:16:57.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:56 smithi157 ceph-mon[116945]: from='osd.9 [v2:172.21.15.120:6826/1402414981,v1:172.21.15.120:6827/1402414981]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2023-06-23T01:16:57.594 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-06-23T01:16:57.595 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-06-23T01:16:57.595 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2023-06-23T01:16:57.595 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2023-06-23T01:16:57.595 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0109368 s, 959 MB/s 2023-06-23T01:16:57.595 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2023-06-23T01:16:58.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:57 smithi120 ceph-mon[79813]: from='osd.9 [v2:172.21.15.120:6826/1402414981,v1:172.21.15.120:6827/1402414981]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2023-06-23T01:16:58.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:57 smithi120 ceph-mon[79813]: osdmap e66: 10 total, 9 up, 10 in 2023-06-23T01:16:58.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:57 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-23T01:16:58.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:57 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-23T01:16:58.193 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:57 smithi134 ceph-mon[116146]: from='osd.9 [v2:172.21.15.120:6826/1402414981,v1:172.21.15.120:6827/1402414981]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2023-06-23T01:16:58.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:57 smithi134 ceph-mon[116146]: osdmap e66: 10 total, 9 up, 10 in 2023-06-23T01:16:58.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:57 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-23T01:16:58.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:57 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-23T01:16:58.278 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph orch daemon add osd smithi134:vg_nvme/lv_1 2023-06-23T01:16:58.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:57 smithi157 ceph-mon[116945]: from='osd.9 [v2:172.21.15.120:6826/1402414981,v1:172.21.15.120:6827/1402414981]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2023-06-23T01:16:58.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:57 smithi157 ceph-mon[116945]: osdmap e66: 10 total, 9 up, 10 in 2023-06-23T01:16:58.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:57 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-23T01:16:58.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:57 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-23T01:16:58.855 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.b/config 2023-06-23T01:16:59.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:58 smithi134 ceph-mon[116146]: purged_snaps scrub starts 2023-06-23T01:16:59.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:58 smithi134 ceph-mon[116146]: purged_snaps scrub ok 2023-06-23T01:16:59.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:58 smithi134 ceph-mon[116146]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-06-23T01:16:59.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:58 smithi134 ceph-mon[116146]: osd.9 [v2:172.21.15.120:6826/1402414981,v1:172.21.15.120:6827/1402414981] boot 2023-06-23T01:16:59.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:58 smithi134 ceph-mon[116146]: osdmap e67: 10 total, 10 up, 10 in 2023-06-23T01:16:59.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:58 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-23T01:16:59.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:58 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:59.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:58 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:59.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:58 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:16:59.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:58 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:16:59.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:58 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:16:59.196 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:58 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:16:59.196 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:58 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:59.196 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:58 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:16:59.196 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:58 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:16:59.196 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:58 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:59.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:58 smithi120 ceph-mon[79813]: purged_snaps scrub starts 2023-06-23T01:16:59.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:58 smithi120 ceph-mon[79813]: purged_snaps scrub ok 2023-06-23T01:16:59.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:58 smithi120 ceph-mon[79813]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-06-23T01:16:59.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:58 smithi120 ceph-mon[79813]: osd.9 [v2:172.21.15.120:6826/1402414981,v1:172.21.15.120:6827/1402414981] boot 2023-06-23T01:16:59.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:58 smithi120 ceph-mon[79813]: osdmap e67: 10 total, 10 up, 10 in 2023-06-23T01:16:59.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:58 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-23T01:16:59.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:58 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:59.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:58 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:59.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:58 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:16:59.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:58 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:16:59.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:58 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:16:59.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:58 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:16:59.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:58 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:59.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:58 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:16:59.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:58 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:16:59.263 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:58 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:59.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:58 smithi157 ceph-mon[116945]: purged_snaps scrub starts 2023-06-23T01:16:59.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:58 smithi157 ceph-mon[116945]: purged_snaps scrub ok 2023-06-23T01:16:59.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:58 smithi157 ceph-mon[116945]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-06-23T01:16:59.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:58 smithi157 ceph-mon[116945]: osd.9 [v2:172.21.15.120:6826/1402414981,v1:172.21.15.120:6827/1402414981] boot 2023-06-23T01:16:59.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:58 smithi157 ceph-mon[116945]: osdmap e67: 10 total, 10 up, 10 in 2023-06-23T01:16:59.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:58 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-23T01:16:59.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:58 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:59.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:58 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:59.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:58 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:16:59.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:58 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:16:59.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:58 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:16:59.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:58 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:16:59.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:58 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:16:59.379 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:58 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:16:59.379 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:58 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:16:59.379 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:58 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:00.097 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:59 smithi134 ceph-mon[116146]: Detected new or changed devices on smithi120 2023-06-23T01:17:00.098 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:59 smithi134 ceph-mon[116146]: Adjusting osd_memory_target on smithi120 to 4316M 2023-06-23T01:17:00.098 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:16:59 smithi134 ceph-mon[116146]: osdmap e68: 10 total, 10 up, 10 in 2023-06-23T01:17:00.236 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.233+0000 7f94ba948700 1 -- 172.21.15.134:0/2660047569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94b40fde40 msgr2=0x7f94b40fe290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:17:00.236 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.233+0000 7f94ba948700 1 --2- 172.21.15.134:0/2660047569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94b40fde40 0x7f94b40fe290 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f94a80049c0 tx=0x7f94a8009e30 comp rx=0 tx=0).stop 2023-06-23T01:17:00.236 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.233+0000 7f94ba948700 1 -- 172.21.15.134:0/2660047569 shutdown_connections 2023-06-23T01:17:00.236 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.233+0000 7f94ba948700 1 --2- 172.21.15.134:0/2660047569 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94b40fe7d0 0x7f94b4105090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:00.236 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.233+0000 7f94ba948700 1 --2- 172.21.15.134:0/2660047569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94b40fcc30 0x7f94b40fd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:00.236 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.233+0000 7f94ba948700 1 --2- 172.21.15.134:0/2660047569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94b40fde40 0x7f94b40fe290 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:00.237 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.233+0000 7f94ba948700 1 -- 172.21.15.134:0/2660047569 >> 172.21.15.134:0/2660047569 conn(0x7f94b40f8350 msgr2=0x7f94b40fa7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:17:00.237 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.233+0000 7f94ba948700 1 -- 172.21.15.134:0/2660047569 shutdown_connections 2023-06-23T01:17:00.237 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.233+0000 7f94ba948700 1 -- 172.21.15.134:0/2660047569 wait complete. 2023-06-23T01:17:00.237 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.233+0000 7f94ba948700 1 Processor -- start 2023-06-23T01:17:00.237 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.233+0000 7f94ba948700 1 -- start start 2023-06-23T01:17:00.239 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.234+0000 7f94ba948700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94b40fcc30 0x7f94b410d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:17:00.239 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.234+0000 7f94ba948700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94b40fde40 0x7f94b410dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:17:00.240 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.234+0000 7f94ba948700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94b40fe7d0 0x7f94b4114e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:17:00.240 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.234+0000 7f94ba948700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b4115430 con 0x7f94b40fcc30 2023-06-23T01:17:00.240 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.234+0000 7f94ba948700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b41155a0 con 0x7f94b40fde40 2023-06-23T01:17:00.240 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.234+0000 7f94ba948700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b4115760 con 0x7f94b40fe7d0 2023-06-23T01:17:00.240 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.234+0000 7f94b37fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94b40fde40 0x7f94b410dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:17:00.240 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.234+0000 7f94b37fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94b40fde40 0x7f94b410dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:43426/0 (socket says 172.21.15.134:43426) 2023-06-23T01:17:00.240 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.234+0000 7f94b37fe700 1 -- 172.21.15.134:0/2494396569 learned_addr learned my addr 172.21.15.134:0/2494396569 (peer_addr_for_me v2:172.21.15.134:0/0) 2023-06-23T01:17:00.241 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.234+0000 7f94b8ee5700 1 --2- 172.21.15.134:0/2494396569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94b40fe7d0 0x7f94b4114e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:17:00.241 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.234+0000 7f94b37fe700 1 -- 172.21.15.134:0/2494396569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94b40fe7d0 msgr2=0x7f94b4114e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:17:00.241 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.234+0000 7f94b37fe700 1 --2- 172.21.15.134:0/2494396569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94b40fe7d0 0x7f94b4114e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:00.241 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.234+0000 7f94b37fe700 1 -- 172.21.15.134:0/2494396569 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94b40fcc30 msgr2=0x7f94b410d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:17:00.241 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.234+0000 7f94b37fe700 1 --2- 172.21.15.134:0/2494396569 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94b40fcc30 0x7f94b410d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:00.241 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.234+0000 7f94b37fe700 1 -- 172.21.15.134:0/2494396569 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a8005040 con 0x7f94b40fde40 2023-06-23T01:17:00.241 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.234+0000 7f94b37fe700 1 --2- 172.21.15.134:0/2494396569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94b40fde40 0x7f94b410dd90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f94a8007f30 tx=0x7f94a8009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:17:00.242 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.234+0000 7f94b0ff9700 1 -- 172.21.15.134:0/2494396569 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a800e040 con 0x7f94b40fde40 2023-06-23T01:17:00.242 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.234+0000 7f94ba948700 1 -- 172.21.15.134:0/2494396569 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94b41159e0 con 0x7f94b40fde40 2023-06-23T01:17:00.242 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.234+0000 7f94b0ff9700 1 -- 172.21.15.134:0/2494396569 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94a8009770 con 0x7f94b40fde40 2023-06-23T01:17:00.242 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.234+0000 7f94b0ff9700 1 -- 172.21.15.134:0/2494396569 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a8013ea0 con 0x7f94b40fde40 2023-06-23T01:17:00.242 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.234+0000 7f94ba948700 1 -- 172.21.15.134:0/2494396569 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94b4115e90 con 0x7f94b40fde40 2023-06-23T01:17:00.242 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.235+0000 7f94ba948700 1 -- 172.21.15.134:0/2494396569 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94b4104110 con 0x7f94b40fde40 2023-06-23T01:17:00.242 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.236+0000 7f94b0ff9700 1 -- 172.21.15.134:0/2494396569 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7f94a8013590 con 0x7f94b40fde40 2023-06-23T01:17:00.243 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.237+0000 7f94b0ff9700 1 --2- 172.21.15.134:0/2494396569 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f949c09b680 0x7f949c09db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:17:00.243 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.237+0000 7f94b0ff9700 1 -- 172.21.15.134:0/2494396569 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6257+0+0 (secure 0 0 0) 0x7f94a80c6320 con 0x7f94b40fde40 2023-06-23T01:17:00.243 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.237+0000 7f94b3fff700 1 --2- 172.21.15.134:0/2494396569 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f949c09b680 0x7f949c09db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:17:00.243 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.238+0000 7f94b3fff700 1 --2- 172.21.15.134:0/2494396569 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f949c09b680 0x7f949c09db30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f94b40fdca0 tx=0x7f94a4009ce0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:17:00.244 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.240+0000 7f94b0ff9700 1 -- 172.21.15.134:0/2494396569 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f94a8092380 con 0x7f94b40fde40 2023-06-23T01:17:00.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:59 smithi120 ceph-mon[79813]: Detected new or changed devices on smithi120 2023-06-23T01:17:00.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:59 smithi120 ceph-mon[79813]: Adjusting osd_memory_target on smithi120 to 4316M 2023-06-23T01:17:00.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:16:59 smithi120 ceph-mon[79813]: osdmap e68: 10 total, 10 up, 10 in 2023-06-23T01:17:00.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:59 smithi157 ceph-mon[116945]: Detected new or changed devices on smithi120 2023-06-23T01:17:00.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:59 smithi157 ceph-mon[116945]: Adjusting osd_memory_target on smithi120 to 4316M 2023-06-23T01:17:00.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:16:59 smithi157 ceph-mon[116945]: osdmap e68: 10 total, 10 up, 10 in 2023-06-23T01:17:00.436 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:00.433+0000 7f94ba948700 1 -- 172.21.15.134:0/2494396569 --> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f94b402b1a0 con 0x7f949c09b680 2023-06-23T01:17:01.232 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:00 smithi134 ceph-mon[116146]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-06-23T01:17:01.232 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:00 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T01:17:01.233 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:00 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T01:17:01.233 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:00 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:17:01.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:00 smithi120 ceph-mon[79813]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-06-23T01:17:01.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:00 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T01:17:01.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:00 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T01:17:01.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:00 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:17:01.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:00 smithi157 ceph-mon[116945]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-06-23T01:17:01.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:00 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T01:17:01.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:00 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T01:17:01.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:00 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:17:02.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:01 smithi120 ceph-mon[79813]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:17:02.281 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:01 smithi134 ceph-mon[116146]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:17:02.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:01 smithi157 ceph-mon[116945]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:17:03.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:02 smithi120 ceph-mon[79813]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-06-23T01:17:03.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:02 smithi157 ceph-mon[116945]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-06-23T01:17:03.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:02 smithi134 ceph-mon[116146]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-06-23T01:17:04.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:04 smithi157 ceph-mon[116945]: from='client.? 172.21.15.134:0/1393356614' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "777f3add-3779-4221-a86f-65cca2e5e8ae"}]: dispatch 2023-06-23T01:17:04.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:04 smithi157 ceph-mon[116945]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "777f3add-3779-4221-a86f-65cca2e5e8ae"}]: dispatch 2023-06-23T01:17:04.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:04 smithi157 ceph-mon[116945]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "777f3add-3779-4221-a86f-65cca2e5e8ae"}]': finished 2023-06-23T01:17:04.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:04 smithi157 ceph-mon[116945]: osdmap e69: 11 total, 10 up, 11 in 2023-06-23T01:17:04.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:04 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-23T01:17:04.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:04 smithi157 ceph-mon[116945]: from='client.? 172.21.15.134:0/2491834585' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T01:17:04.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:04 smithi134 ceph-mon[116146]: from='client.? 172.21.15.134:0/1393356614' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "777f3add-3779-4221-a86f-65cca2e5e8ae"}]: dispatch 2023-06-23T01:17:04.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:04 smithi134 ceph-mon[116146]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "777f3add-3779-4221-a86f-65cca2e5e8ae"}]: dispatch 2023-06-23T01:17:04.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:04 smithi134 ceph-mon[116146]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "777f3add-3779-4221-a86f-65cca2e5e8ae"}]': finished 2023-06-23T01:17:04.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:04 smithi134 ceph-mon[116146]: osdmap e69: 11 total, 10 up, 11 in 2023-06-23T01:17:04.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:04 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-23T01:17:04.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:04 smithi134 ceph-mon[116146]: from='client.? 172.21.15.134:0/2491834585' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T01:17:04.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:04 smithi120 ceph-mon[79813]: from='client.? 172.21.15.134:0/1393356614' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "777f3add-3779-4221-a86f-65cca2e5e8ae"}]: dispatch 2023-06-23T01:17:04.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:04 smithi120 ceph-mon[79813]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "777f3add-3779-4221-a86f-65cca2e5e8ae"}]: dispatch 2023-06-23T01:17:04.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:04 smithi120 ceph-mon[79813]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "777f3add-3779-4221-a86f-65cca2e5e8ae"}]': finished 2023-06-23T01:17:04.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:04 smithi120 ceph-mon[79813]: osdmap e69: 11 total, 10 up, 11 in 2023-06-23T01:17:04.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:04 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-23T01:17:04.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:04 smithi120 ceph-mon[79813]: from='client.? 172.21.15.134:0/2491834585' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T01:17:05.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:05 smithi157 ceph-mon[116945]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-06-23T01:17:05.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:05 smithi134 ceph-mon[116146]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-06-23T01:17:05.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:05 smithi120 ceph-mon[79813]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-06-23T01:17:06.383 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:06 smithi157 ceph-mon[116945]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-06-23T01:17:06.454 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:06 smithi134 ceph-mon[116146]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-06-23T01:17:06.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:06 smithi120 ceph-mon[79813]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-06-23T01:17:09.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:08 smithi134 ceph-mon[116146]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-06-23T01:17:09.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:08 smithi120 ceph-mon[79813]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-06-23T01:17:09.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:08 smithi157 ceph-mon[116945]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-06-23T01:17:11.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:10 smithi120 ceph-mon[79813]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-06-23T01:17:11.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:10 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-06-23T01:17:11.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:10 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:17:11.331 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:10 smithi134 ceph-mon[116146]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-06-23T01:17:11.332 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:10 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-06-23T01:17:11.332 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:10 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:17:11.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:10 smithi157 ceph-mon[116945]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-06-23T01:17:11.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:10 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-06-23T01:17:11.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:10 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:17:12.247 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:11 smithi134 ceph-mon[116146]: Deploying daemon osd.10 on smithi134 2023-06-23T01:17:12.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:11 smithi120 ceph-mon[79813]: Deploying daemon osd.10 on smithi134 2023-06-23T01:17:12.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:11 smithi157 ceph-mon[116945]: Deploying daemon osd.10 on smithi134 2023-06-23T01:17:13.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:12 smithi120 ceph-mon[79813]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-06-23T01:17:13.337 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:12 smithi134 ceph-mon[116146]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-06-23T01:17:13.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:12 smithi157 ceph-mon[116945]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-06-23T01:17:15.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:14 smithi120 ceph-mon[79813]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-06-23T01:17:15.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:14 smithi157 ceph-mon[116945]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-06-23T01:17:15.387 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:14 smithi134 ceph-mon[116146]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-06-23T01:17:16.694 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:16 smithi134 ceph-mon[116146]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-06-23T01:17:16.694 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:16 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:16.695 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:16 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:16.695 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:16 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:17:16.876 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:16 smithi157 ceph-mon[116945]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-06-23T01:17:16.876 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:16 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:16.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:16 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:16.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:16 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:17:17.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:16 smithi120 ceph-mon[79813]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-06-23T01:17:17.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:16 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:17.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:16 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:17.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:16 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:17:19.004 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 10 on host 'smithi134' 2023-06-23T01:17:19.004 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:19.000+0000 7f94b0ff9700 1 -- 172.21.15.134:0/2494396569 <== mgr.14152 v2:172.21.15.120:6800/1770912857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f94b402b1a0 con 0x7f949c09b680 2023-06-23T01:17:19.005 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:19.002+0000 7f94ba948700 1 -- 172.21.15.134:0/2494396569 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f949c09b680 msgr2=0x7f949c09db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:17:19.006 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:19.002+0000 7f94ba948700 1 --2- 172.21.15.134:0/2494396569 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f949c09b680 0x7f949c09db30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f94b40fdca0 tx=0x7f94a4009ce0 comp rx=0 tx=0).stop 2023-06-23T01:17:19.006 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:19.002+0000 7f94ba948700 1 -- 172.21.15.134:0/2494396569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94b40fde40 msgr2=0x7f94b410dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:17:19.006 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:19.002+0000 7f94ba948700 1 --2- 172.21.15.134:0/2494396569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94b40fde40 0x7f94b410dd90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f94a8007f30 tx=0x7f94a8009e30 comp rx=0 tx=0).stop 2023-06-23T01:17:19.006 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:19.002+0000 7f94ba948700 1 -- 172.21.15.134:0/2494396569 shutdown_connections 2023-06-23T01:17:19.007 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:19.002+0000 7f94ba948700 1 --2- 172.21.15.134:0/2494396569 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f949c09b680 0x7f949c09db30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:19.007 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:19.002+0000 7f94ba948700 1 --2- 172.21.15.134:0/2494396569 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94b40fcc30 0x7f94b410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:19.007 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:19.002+0000 7f94ba948700 1 --2- 172.21.15.134:0/2494396569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94b40fe7d0 0x7f94b4114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:19.007 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:19.002+0000 7f94ba948700 1 --2- 172.21.15.134:0/2494396569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94b40fde40 0x7f94b410dd90 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:19.007 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:19.002+0000 7f94ba948700 1 -- 172.21.15.134:0/2494396569 >> 172.21.15.134:0/2494396569 conn(0x7f94b40f8350 msgr2=0x7f94b4101290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:17:19.007 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:19.003+0000 7f94ba948700 1 -- 172.21.15.134:0/2494396569 shutdown_connections 2023-06-23T01:17:19.007 INFO:teuthology.orchestra.run.smithi134.stderr:2023-06-23T01:17:19.003+0000 7f94ba948700 1 -- 172.21.15.134:0/2494396569 wait complete. 2023-06-23T01:17:19.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:18 smithi120 ceph-mon[79813]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-06-23T01:17:19.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:18 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:19.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:18 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:19.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:18 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:17:19.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:18 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:17:19.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:18 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:19.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:18 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:19.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:18 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:17:19.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:18 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:19.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:18 smithi157 ceph-mon[116945]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-06-23T01:17:19.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:18 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:19.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:18 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:19.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:18 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:17:19.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:18 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:17:19.379 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:18 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:19.379 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:18 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:19.379 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:18 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:17:19.379 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:18 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:19.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:18 smithi134 ceph-mon[116146]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-06-23T01:17:19.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:18 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:19.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:18 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:19.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:18 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:17:19.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:18 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:17:19.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:18 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:19.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:18 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:19.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:18 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:17:19.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:18 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:19.616 DEBUG:teuthology.orchestra.run.smithi134:osd.10> sudo journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@osd.10.service 2023-06-23T01:17:19.618 INFO:tasks.cephadm:Deploying osd.11 on smithi157 with /dev/vg_nvme/lv_1... 2023-06-23T01:17:19.618 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-06-23T01:17:19.646 INFO:journalctl@ceph.osd.10.smithi134.stdout:-- Logs begin at Fri 2023-06-23 00:49:19 UTC. -- 2023-06-23T01:17:20.212 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.c/config 2023-06-23T01:17:20.237 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:19 smithi134 ceph-mon[116146]: from='osd.10 [v2:172.21.15.134:6824/4041441144,v1:172.21.15.134:6825/4041441144]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-06-23T01:17:20.237 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:19 smithi134 ceph-mon[116146]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-06-23T01:17:20.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:19 smithi120 ceph-mon[79813]: from='osd.10 [v2:172.21.15.134:6824/4041441144,v1:172.21.15.134:6825/4041441144]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-06-23T01:17:20.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:19 smithi120 ceph-mon[79813]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-06-23T01:17:20.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:19 smithi157 ceph-mon[116945]: from='osd.10 [v2:172.21.15.134:6824/4041441144,v1:172.21.15.134:6825/4041441144]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-06-23T01:17:20.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:19 smithi157 ceph-mon[116945]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-06-23T01:17:21.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:20 smithi157 ceph-mon[116945]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-06-23T01:17:21.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:20 smithi157 ceph-mon[116945]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-06-23T01:17:21.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:20 smithi157 ceph-mon[116945]: osdmap e70: 11 total, 10 up, 11 in 2023-06-23T01:17:21.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:20 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-23T01:17:21.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:20 smithi157 ceph-mon[116945]: from='osd.10 [v2:172.21.15.134:6824/4041441144,v1:172.21.15.134:6825/4041441144]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-06-23T01:17:21.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:20 smithi157 ceph-mon[116945]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-06-23T01:17:21.146 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:20 smithi134 ceph-mon[116146]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-06-23T01:17:21.147 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:20 smithi134 ceph-mon[116146]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-06-23T01:17:21.147 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:20 smithi134 ceph-mon[116146]: osdmap e70: 11 total, 10 up, 11 in 2023-06-23T01:17:21.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:20 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-23T01:17:21.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:20 smithi134 ceph-mon[116146]: from='osd.10 [v2:172.21.15.134:6824/4041441144,v1:172.21.15.134:6825/4041441144]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-06-23T01:17:21.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:20 smithi134 ceph-mon[116146]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-06-23T01:17:21.148 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:17:20 smithi134 conmon[143713]: 2023-06-23T01:17:20.991+0000 7ffba5c4a700 -1 osd.10 0 waiting for initial osdmap 2023-06-23T01:17:21.148 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:17:21 smithi134 conmon[143713]: 2023-06-23T01:17:21.011+0000 7ffba1a47700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T01:17:21.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:20 smithi120 ceph-mon[79813]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-06-23T01:17:21.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:20 smithi120 ceph-mon[79813]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-06-23T01:17:21.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:20 smithi120 ceph-mon[79813]: osdmap e70: 11 total, 10 up, 11 in 2023-06-23T01:17:21.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:20 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-23T01:17:21.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:20 smithi120 ceph-mon[79813]: from='osd.10 [v2:172.21.15.134:6824/4041441144,v1:172.21.15.134:6825/4041441144]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-06-23T01:17:21.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:20 smithi120 ceph-mon[79813]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-06-23T01:17:21.973 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-06-23T01:17:21.974 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-06-23T01:17:21.974 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2023-06-23T01:17:21.974 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2023-06-23T01:17:21.974 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00829895 s, 1.3 GB/s 2023-06-23T01:17:21.974 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2023-06-23T01:17:22.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:21 smithi120 ceph-mon[79813]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-06-23T01:17:22.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:21 smithi120 ceph-mon[79813]: osdmap e71: 11 total, 10 up, 11 in 2023-06-23T01:17:22.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:21 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-23T01:17:22.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:21 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-23T01:17:22.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:21 smithi120 ceph-mon[79813]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-06-23T01:17:22.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:21 smithi120 ceph-mon[79813]: osd.10 [v2:172.21.15.134:6824/4041441144,v1:172.21.15.134:6825/4041441144] boot 2023-06-23T01:17:22.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:21 smithi120 ceph-mon[79813]: osdmap e72: 11 total, 11 up, 11 in 2023-06-23T01:17:22.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:21 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-23T01:17:22.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:21 smithi157 ceph-mon[116945]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-06-23T01:17:22.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:21 smithi157 ceph-mon[116945]: osdmap e71: 11 total, 10 up, 11 in 2023-06-23T01:17:22.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:21 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-23T01:17:22.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:21 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-23T01:17:22.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:21 smithi157 ceph-mon[116945]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-06-23T01:17:22.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:21 smithi157 ceph-mon[116945]: osd.10 [v2:172.21.15.134:6824/4041441144,v1:172.21.15.134:6825/4041441144] boot 2023-06-23T01:17:22.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:21 smithi157 ceph-mon[116945]: osdmap e72: 11 total, 11 up, 11 in 2023-06-23T01:17:22.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:21 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-23T01:17:22.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:21 smithi134 ceph-mon[116146]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-06-23T01:17:22.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:21 smithi134 ceph-mon[116146]: osdmap e71: 11 total, 10 up, 11 in 2023-06-23T01:17:22.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:21 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-23T01:17:22.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:21 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-23T01:17:22.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:21 smithi134 ceph-mon[116146]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-06-23T01:17:22.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:21 smithi134 ceph-mon[116146]: osd.10 [v2:172.21.15.134:6824/4041441144,v1:172.21.15.134:6825/4041441144] boot 2023-06-23T01:17:22.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:21 smithi134 ceph-mon[116146]: osdmap e72: 11 total, 11 up, 11 in 2023-06-23T01:17:22.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:21 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-23T01:17:22.647 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph orch daemon add osd smithi157:vg_nvme/lv_1 2023-06-23T01:17:23.231 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.c/config 2023-06-23T01:17:23.660 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:23 smithi157 ceph-mon[116945]: purged_snaps scrub starts 2023-06-23T01:17:23.660 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:23 smithi157 ceph-mon[116945]: purged_snaps scrub ok 2023-06-23T01:17:23.660 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:23 smithi157 ceph-mon[116945]: Detected new or changed devices on smithi134 2023-06-23T01:17:23.660 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:23 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:23.661 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:23 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:23.661 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:23 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:17:23.661 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:23 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:17:23.661 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:23 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:17:23.661 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:23 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:17:23.661 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:23 smithi157 ceph-mon[116945]: Adjusting osd_memory_target on smithi134 to 4293M 2023-06-23T01:17:23.661 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:23 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:23.661 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:23 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:17:23.662 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:23 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:17:23.662 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:23 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:23.662 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:23 smithi157 ceph-mon[116945]: osdmap e73: 11 total, 11 up, 11 in 2023-06-23T01:17:23.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:23 smithi134 ceph-mon[116146]: purged_snaps scrub starts 2023-06-23T01:17:23.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:23 smithi134 ceph-mon[116146]: purged_snaps scrub ok 2023-06-23T01:17:23.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:23 smithi134 ceph-mon[116146]: Detected new or changed devices on smithi134 2023-06-23T01:17:23.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:23 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:23.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:23 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:23.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:23 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:17:23.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:23 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:17:23.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:23 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:17:23.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:23 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:17:23.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:23 smithi134 ceph-mon[116146]: Adjusting osd_memory_target on smithi134 to 4293M 2023-06-23T01:17:23.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:23 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:23.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:23 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:17:23.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:23 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:17:23.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:23 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:23.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:23 smithi134 ceph-mon[116146]: osdmap e73: 11 total, 11 up, 11 in 2023-06-23T01:17:24.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:23 smithi120 ceph-mon[79813]: purged_snaps scrub starts 2023-06-23T01:17:24.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:23 smithi120 ceph-mon[79813]: purged_snaps scrub ok 2023-06-23T01:17:24.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:23 smithi120 ceph-mon[79813]: Detected new or changed devices on smithi134 2023-06-23T01:17:24.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:23 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:24.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:23 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:24.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:23 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:17:24.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:23 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:17:24.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:23 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:17:24.013 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:23 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:17:24.013 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:23 smithi120 ceph-mon[79813]: Adjusting osd_memory_target on smithi134 to 4293M 2023-06-23T01:17:24.013 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:23 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:24.013 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:23 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:17:24.014 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:23 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:17:24.014 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:23 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:24.014 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:23 smithi120 ceph-mon[79813]: osdmap e73: 11 total, 11 up, 11 in 2023-06-23T01:17:24.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.717+0000 7feecb298700 1 -- 172.21.15.157:0/3344502833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feec40fe840 msgr2=0x7feec4105100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:17:24.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.717+0000 7feecb298700 1 --2- 172.21.15.157:0/3344502833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feec40fe840 0x7feec4105100 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7feebc009d60 tx=0x7feebc004d10 comp rx=0 tx=0).stop 2023-06-23T01:17:24.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.717+0000 7feecb298700 1 -- 172.21.15.157:0/3344502833 shutdown_connections 2023-06-23T01:17:24.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.717+0000 7feecb298700 1 --2- 172.21.15.157:0/3344502833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feec40fcca0 0x7feec40fd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:24.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.717+0000 7feecb298700 1 --2- 172.21.15.157:0/3344502833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feec40fe840 0x7feec4105100 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:24.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.717+0000 7feecb298700 1 --2- 172.21.15.157:0/3344502833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feec40fdeb0 0x7feec40fe300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:24.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.717+0000 7feecb298700 1 -- 172.21.15.157:0/3344502833 >> 172.21.15.157:0/3344502833 conn(0x7feec40f8400 msgr2=0x7feec40fa830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:17:24.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.717+0000 7feecb298700 1 -- 172.21.15.157:0/3344502833 shutdown_connections 2023-06-23T01:17:24.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.717+0000 7feecb298700 1 -- 172.21.15.157:0/3344502833 wait complete. 2023-06-23T01:17:24.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.717+0000 7feecb298700 1 Processor -- start 2023-06-23T01:17:24.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.717+0000 7feecb298700 1 -- start start 2023-06-23T01:17:24.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.717+0000 7feecb298700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feec40fcca0 0x7feec410d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:17:24.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.717+0000 7feecb298700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feec40fdeb0 0x7feec410dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:17:24.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.717+0000 7feecb298700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feec40fe840 0x7feec4114d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:17:24.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.717+0000 7feecb298700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec4115390 con 0x7feec40fdeb0 2023-06-23T01:17:24.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.717+0000 7feecb298700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec4115500 con 0x7feec40fe840 2023-06-23T01:17:24.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.717+0000 7feecb298700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec41156c0 con 0x7feec40fcca0 2023-06-23T01:17:24.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.718+0000 7feec9034700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feec40fcca0 0x7feec410d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:17:24.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.718+0000 7feec9034700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feec40fcca0 0x7feec410d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39322/0 (socket says 172.21.15.157:39322) 2023-06-23T01:17:24.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.718+0000 7feec9034700 1 -- 172.21.15.157:0/1430919538 learned_addr learned my addr 172.21.15.157:0/1430919538 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-06-23T01:17:24.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.718+0000 7feec9835700 1 --2- 172.21.15.157:0/1430919538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feec40fe840 0x7feec4114d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:17:24.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.718+0000 7feec9034700 1 -- 172.21.15.157:0/1430919538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feec40fe840 msgr2=0x7feec4114d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:17:24.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.718+0000 7feec9034700 1 --2- 172.21.15.157:0/1430919538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feec40fe840 0x7feec4114d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:24.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.718+0000 7feec9034700 1 -- 172.21.15.157:0/1430919538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feec40fdeb0 msgr2=0x7feec410dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:17:24.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.718+0000 7feec9034700 1 --2- 172.21.15.157:0/1430919538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feec40fdeb0 0x7feec410dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:24.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.718+0000 7feec9034700 1 -- 172.21.15.157:0/1430919538 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feebc009940 con 0x7feec40fcca0 2023-06-23T01:17:24.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.718+0000 7feec9034700 1 --2- 172.21.15.157:0/1430919538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feec40fcca0 0x7feec410d7b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7feec000b810 tx=0x7feec0009220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:17:24.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.718+0000 7feeb9ffb700 1 -- 172.21.15.157:0/1430919538 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec000f5f0 con 0x7feec40fcca0 2023-06-23T01:17:24.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.718+0000 7feeb9ffb700 1 -- 172.21.15.157:0/1430919538 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feec000fc20 con 0x7feec40fcca0 2023-06-23T01:17:24.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.718+0000 7feecb298700 1 -- 172.21.15.157:0/1430919538 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feec41159a0 con 0x7feec40fcca0 2023-06-23T01:17:24.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.718+0000 7feeb9ffb700 1 -- 172.21.15.157:0/1430919538 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec0010cc0 con 0x7feec40fcca0 2023-06-23T01:17:24.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.718+0000 7feecb298700 1 -- 172.21.15.157:0/1430919538 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec4115e90 con 0x7feec40fcca0 2023-06-23T01:17:24.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.719+0000 7feecb298700 1 -- 172.21.15.157:0/1430919538 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feec4104040 con 0x7feec40fcca0 2023-06-23T01:17:24.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.720+0000 7feeb9ffb700 1 -- 172.21.15.157:0/1430919538 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7feec000fd90 con 0x7feec40fcca0 2023-06-23T01:17:24.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.720+0000 7feeb9ffb700 1 --2- 172.21.15.157:0/1430919538 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7feeb009b680 0x7feeb009db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:17:24.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.721+0000 7feeb9ffb700 1 -- 172.21.15.157:0/1430919538 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6719+0+0 (secure 0 0 0) 0x7feec00c4780 con 0x7feec40fcca0 2023-06-23T01:17:24.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.721+0000 7feec8833700 1 --2- 172.21.15.157:0/1430919538 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7feeb009b680 0x7feeb009db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:17:24.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.721+0000 7feec8833700 1 --2- 172.21.15.157:0/1430919538 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7feeb009b680 0x7feeb009db30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7feeb4004ad0 tx=0x7feeb4007d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:17:24.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.725+0000 7feeb9ffb700 1 -- 172.21.15.157:0/1430919538 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7feec0090620 con 0x7feec40fcca0 2023-06-23T01:17:24.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:24.917+0000 7feecb298700 1 -- 172.21.15.157:0/1430919538 --> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7feec402b1b0 con 0x7feeb009b680 2023-06-23T01:17:25.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:24 smithi120 ceph-mon[79813]: pgmap v235: 1 pgs: 1 remapped+peering; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-06-23T01:17:25.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:24 smithi120 ceph-mon[79813]: osdmap e74: 11 total, 11 up, 11 in 2023-06-23T01:17:25.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:24 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T01:17:25.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:24 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T01:17:25.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:24 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:17:25.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:24 smithi157 ceph-mon[116945]: pgmap v235: 1 pgs: 1 remapped+peering; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-06-23T01:17:25.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:24 smithi157 ceph-mon[116945]: osdmap e74: 11 total, 11 up, 11 in 2023-06-23T01:17:25.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:24 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T01:17:25.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:24 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T01:17:25.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:24 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:17:25.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:24 smithi134 ceph-mon[116146]: pgmap v235: 1 pgs: 1 remapped+peering; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-06-23T01:17:25.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:24 smithi134 ceph-mon[116146]: osdmap e74: 11 total, 11 up, 11 in 2023-06-23T01:17:25.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:24 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T01:17:25.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:24 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T01:17:25.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:24 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:17:26.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:25 smithi120 ceph-mon[79813]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:17:26.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:25 smithi120 ceph-mon[79813]: osdmap e75: 11 total, 11 up, 11 in 2023-06-23T01:17:26.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:25 smithi157 ceph-mon[116945]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:17:26.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:25 smithi157 ceph-mon[116945]: osdmap e75: 11 total, 11 up, 11 in 2023-06-23T01:17:26.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:25 smithi134 ceph-mon[116146]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:17:26.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:25 smithi134 ceph-mon[116146]: osdmap e75: 11 total, 11 up, 11 in 2023-06-23T01:17:27.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:26 smithi157 ceph-mon[116945]: pgmap v238: 1 pgs: 1 remapped+peering; 577 KiB data, 68 MiB used, 983 GiB / 983 GiB avail 2023-06-23T01:17:27.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:26 smithi157 ceph-mon[116945]: osdmap e76: 11 total, 11 up, 11 in 2023-06-23T01:17:27.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:26 smithi120 ceph-mon[79813]: pgmap v238: 1 pgs: 1 remapped+peering; 577 KiB data, 68 MiB used, 983 GiB / 983 GiB avail 2023-06-23T01:17:27.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:26 smithi120 ceph-mon[79813]: osdmap e76: 11 total, 11 up, 11 in 2023-06-23T01:17:27.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:26 smithi134 ceph-mon[116146]: pgmap v238: 1 pgs: 1 remapped+peering; 577 KiB data, 68 MiB used, 983 GiB / 983 GiB avail 2023-06-23T01:17:27.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:26 smithi134 ceph-mon[116146]: osdmap e76: 11 total, 11 up, 11 in 2023-06-23T01:17:28.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:27 smithi120 ceph-mon[79813]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-06-23T01:17:28.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:27 smithi120 ceph-mon[79813]: from='client.? 172.21.15.157:0/3432217372' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7166e11b-03c0-45bf-abbf-a74c4a13a39d"}]: dispatch 2023-06-23T01:17:28.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:27 smithi120 ceph-mon[79813]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7166e11b-03c0-45bf-abbf-a74c4a13a39d"}]: dispatch 2023-06-23T01:17:28.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:27 smithi120 ceph-mon[79813]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7166e11b-03c0-45bf-abbf-a74c4a13a39d"}]': finished 2023-06-23T01:17:28.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:27 smithi120 ceph-mon[79813]: osdmap e77: 12 total, 11 up, 12 in 2023-06-23T01:17:28.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:27 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-23T01:17:28.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:27 smithi157 ceph-mon[116945]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-06-23T01:17:28.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:27 smithi157 ceph-mon[116945]: from='client.? 172.21.15.157:0/3432217372' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7166e11b-03c0-45bf-abbf-a74c4a13a39d"}]: dispatch 2023-06-23T01:17:28.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:27 smithi157 ceph-mon[116945]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7166e11b-03c0-45bf-abbf-a74c4a13a39d"}]: dispatch 2023-06-23T01:17:28.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:27 smithi157 ceph-mon[116945]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7166e11b-03c0-45bf-abbf-a74c4a13a39d"}]': finished 2023-06-23T01:17:28.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:27 smithi157 ceph-mon[116945]: osdmap e77: 12 total, 11 up, 12 in 2023-06-23T01:17:28.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:27 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-23T01:17:28.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:27 smithi134 ceph-mon[116146]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-06-23T01:17:28.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:27 smithi134 ceph-mon[116146]: from='client.? 172.21.15.157:0/3432217372' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7166e11b-03c0-45bf-abbf-a74c4a13a39d"}]: dispatch 2023-06-23T01:17:28.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:27 smithi134 ceph-mon[116146]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7166e11b-03c0-45bf-abbf-a74c4a13a39d"}]: dispatch 2023-06-23T01:17:28.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:27 smithi134 ceph-mon[116146]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7166e11b-03c0-45bf-abbf-a74c4a13a39d"}]': finished 2023-06-23T01:17:28.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:27 smithi134 ceph-mon[116146]: osdmap e77: 12 total, 11 up, 12 in 2023-06-23T01:17:28.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:27 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-23T01:17:29.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:28 smithi120 ceph-mon[79813]: from='client.? 172.21.15.157:0/4105905428' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T01:17:29.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:28 smithi157 ceph-mon[116945]: from='client.? 172.21.15.157:0/4105905428' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T01:17:29.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:28 smithi134 ceph-mon[116146]: from='client.? 172.21.15.157:0/4105905428' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T01:17:30.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:30 smithi120 ceph-mon[79813]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 68 MiB used, 983 GiB / 983 GiB avail; 114 KiB/s, 0 objects/s recovering 2023-06-23T01:17:30.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:30 smithi157 ceph-mon[116945]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 68 MiB used, 983 GiB / 983 GiB avail; 114 KiB/s, 0 objects/s recovering 2023-06-23T01:17:30.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:30 smithi134 ceph-mon[116146]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 68 MiB used, 983 GiB / 983 GiB avail; 114 KiB/s, 0 objects/s recovering 2023-06-23T01:17:33.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:32 smithi120 ceph-mon[79813]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2023-06-23T01:17:33.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:32 smithi157 ceph-mon[116945]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2023-06-23T01:17:33.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:32 smithi134 ceph-mon[116146]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2023-06-23T01:17:35.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:34 smithi157 ceph-mon[116945]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-06-23T01:17:35.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:34 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-06-23T01:17:35.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:34 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:17:35.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:34 smithi120 ceph-mon[79813]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-06-23T01:17:35.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:34 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-06-23T01:17:35.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:34 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:17:35.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:34 smithi134 ceph-mon[116146]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-06-23T01:17:35.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:34 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-06-23T01:17:35.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:34 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:17:36.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:35 smithi120 ceph-mon[79813]: Deploying daemon osd.11 on smithi157 2023-06-23T01:17:36.284 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:35 smithi157 ceph-mon[116945]: Deploying daemon osd.11 on smithi157 2023-06-23T01:17:36.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:35 smithi134 ceph-mon[116146]: Deploying daemon osd.11 on smithi157 2023-06-23T01:17:37.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:36 smithi120 ceph-mon[79813]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-06-23T01:17:37.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:36 smithi157 ceph-mon[116945]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-06-23T01:17:37.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:36 smithi134 ceph-mon[116146]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-06-23T01:17:38.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:38 smithi157 ceph-mon[116945]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-06-23T01:17:38.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:38 smithi134 ceph-mon[116146]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-06-23T01:17:38.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:38 smithi120 ceph-mon[79813]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-06-23T01:17:41.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:40 smithi157 ceph-mon[116945]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-06-23T01:17:41.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:40 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:41.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:40 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:41.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:40 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:17:41.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:40 smithi120 ceph-mon[79813]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-06-23T01:17:41.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:40 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:41.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:40 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:41.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:40 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:17:41.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:40 smithi134 ceph-mon[116146]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-06-23T01:17:41.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:40 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:41.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:40 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:41.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:40 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:17:43.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:42 smithi120 ceph-mon[79813]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-06-23T01:17:43.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:42 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:43.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:42 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:43.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:42 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:17:43.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:42 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:17:43.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:42 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:43.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:42 smithi157 ceph-mon[116945]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-06-23T01:17:43.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:42 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:43.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:42 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:43.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:42 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:17:43.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:42 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:17:43.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:42 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:43.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:42 smithi134 ceph-mon[116146]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-06-23T01:17:43.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:42 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:43.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:42 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:43.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:42 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:17:43.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:42 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:17:43.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:42 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:43.696 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 11 on host 'smithi157' 2023-06-23T01:17:43.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:43.693+0000 7feeb9ffb700 1 -- 172.21.15.157:0/1430919538 <== mgr.14152 v2:172.21.15.120:6800/1770912857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7feec402b1b0 con 0x7feeb009b680 2023-06-23T01:17:43.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:43.696+0000 7feecb298700 1 -- 172.21.15.157:0/1430919538 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7feeb009b680 msgr2=0x7feeb009db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:17:43.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:43.696+0000 7feecb298700 1 --2- 172.21.15.157:0/1430919538 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7feeb009b680 0x7feeb009db30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7feeb4004ad0 tx=0x7feeb4007d40 comp rx=0 tx=0).stop 2023-06-23T01:17:43.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:43.696+0000 7feecb298700 1 -- 172.21.15.157:0/1430919538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feec40fcca0 msgr2=0x7feec410d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:17:43.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:43.696+0000 7feecb298700 1 --2- 172.21.15.157:0/1430919538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feec40fcca0 0x7feec410d7b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7feec000b810 tx=0x7feec0009220 comp rx=0 tx=0).stop 2023-06-23T01:17:43.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:43.696+0000 7feecb298700 1 -- 172.21.15.157:0/1430919538 shutdown_connections 2023-06-23T01:17:43.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:43.696+0000 7feecb298700 1 --2- 172.21.15.157:0/1430919538 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7feeb009b680 0x7feeb009db30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:43.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:43.696+0000 7feecb298700 1 --2- 172.21.15.157:0/1430919538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feec40fe840 0x7feec4114d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:43.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:43.696+0000 7feecb298700 1 --2- 172.21.15.157:0/1430919538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feec40fcca0 0x7feec410d7b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:43.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:43.696+0000 7feecb298700 1 --2- 172.21.15.157:0/1430919538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feec40fdeb0 0x7feec410dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:43.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:43.696+0000 7feecb298700 1 -- 172.21.15.157:0/1430919538 >> 172.21.15.157:0/1430919538 conn(0x7feec40f8400 msgr2=0x7feec41011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:17:43.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:43.696+0000 7feecb298700 1 -- 172.21.15.157:0/1430919538 shutdown_connections 2023-06-23T01:17:43.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:43.696+0000 7feecb298700 1 -- 172.21.15.157:0/1430919538 wait complete. 2023-06-23T01:17:44.283 DEBUG:teuthology.orchestra.run.smithi157:osd.11> sudo journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@osd.11.service 2023-06-23T01:17:44.286 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-06-23T01:17:44.287 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph osd stat -f json 2023-06-23T01:17:44.313 INFO:journalctl@ceph.osd.11.smithi157.stdout:-- Logs begin at Fri 2023-06-23 00:49:55 UTC. -- 2023-06-23T01:17:44.943 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:44 smithi134 ceph-mon[116146]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-06-23T01:17:44.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:44 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:44.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:44 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:44.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:44 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:17:44.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:44 smithi134 ceph-mon[116146]: from='osd.11 [v2:172.21.15.157:6824/1537477260,v1:172.21.15.157:6825/1537477260]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-06-23T01:17:44.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:44 smithi134 ceph-mon[116146]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-06-23T01:17:44.967 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:44 smithi157 ceph-mon[116945]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-06-23T01:17:44.967 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:44 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:44.967 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:44 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:44.967 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:44 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:17:44.968 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:44 smithi157 ceph-mon[116945]: from='osd.11 [v2:172.21.15.157:6824/1537477260,v1:172.21.15.157:6825/1537477260]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-06-23T01:17:44.968 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:44 smithi157 ceph-mon[116945]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-06-23T01:17:45.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:44 smithi120 ceph-mon[79813]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-06-23T01:17:45.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:44 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:45.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:44 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:45.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:44 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:17:45.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:44 smithi120 ceph-mon[79813]: from='osd.11 [v2:172.21.15.157:6824/1537477260,v1:172.21.15.157:6825/1537477260]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-06-23T01:17:45.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:44 smithi120 ceph-mon[79813]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-06-23T01:17:45.055 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T01:17:45.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:45 smithi134 ceph-mon[116146]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-06-23T01:17:45.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:45 smithi134 ceph-mon[116146]: osdmap e78: 12 total, 11 up, 12 in 2023-06-23T01:17:45.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:45 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-23T01:17:45.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:45 smithi134 ceph-mon[116146]: from='osd.11 [v2:172.21.15.157:6824/1537477260,v1:172.21.15.157:6825/1537477260]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-06-23T01:17:45.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:45 smithi134 ceph-mon[116146]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-06-23T01:17:46.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:45 smithi120 ceph-mon[79813]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-06-23T01:17:46.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:45 smithi120 ceph-mon[79813]: osdmap e78: 12 total, 11 up, 12 in 2023-06-23T01:17:46.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:45 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-23T01:17:46.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:45 smithi120 ceph-mon[79813]: from='osd.11 [v2:172.21.15.157:6824/1537477260,v1:172.21.15.157:6825/1537477260]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-06-23T01:17:46.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:45 smithi120 ceph-mon[79813]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-06-23T01:17:46.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:45 smithi157 ceph-mon[116945]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-06-23T01:17:46.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:45 smithi157 ceph-mon[116945]: osdmap e78: 12 total, 11 up, 12 in 2023-06-23T01:17:46.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:45 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-23T01:17:46.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:45 smithi157 ceph-mon[116945]: from='osd.11 [v2:172.21.15.157:6824/1537477260,v1:172.21.15.157:6825/1537477260]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-06-23T01:17:46.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:45 smithi157 ceph-mon[116945]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2023-06-23T01:17:46.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:17:45 smithi157 conmon[145357]: 2023-06-23T01:17:45.674+0000 7f4e06948700 -1 osd.11 0 waiting for initial osdmap 2023-06-23T01:17:46.128 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:17:45 smithi157 conmon[145357]: 2023-06-23T01:17:45.693+0000 7f4e01743700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T01:17:46.420 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.416+0000 7f499bf3b700 1 -- 172.21.15.120:0/2830200118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49940719b0 msgr2=0x7f499406fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:17:46.420 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.416+0000 7f499bf3b700 1 --2- 172.21.15.120:0/2830200118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49940719b0 0x7f499406fe20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f49900049c0 tx=0x7f4990009e30 comp rx=0 tx=0).stop 2023-06-23T01:17:46.420 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.416+0000 7f499bf3b700 1 -- 172.21.15.120:0/2830200118 shutdown_connections 2023-06-23T01:17:46.421 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.416+0000 7f499bf3b700 1 --2- 172.21.15.120:0/2830200118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4994104aa0 0x7f4994106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:46.421 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.416+0000 7f499bf3b700 1 --2- 172.21.15.120:0/2830200118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49940719b0 0x7f499406fe20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:46.421 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.416+0000 7f499bf3b700 1 --2- 172.21.15.120:0/2830200118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4994070360 0x7f49940707b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:46.421 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.416+0000 7f499bf3b700 1 -- 172.21.15.120:0/2830200118 >> 172.21.15.120:0/2830200118 conn(0x7f49940f8320 msgr2=0x7f49940fa790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:17:46.422 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.416+0000 7f499bf3b700 1 -- 172.21.15.120:0/2830200118 shutdown_connections 2023-06-23T01:17:46.422 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.417+0000 7f499bf3b700 1 -- 172.21.15.120:0/2830200118 wait complete. 2023-06-23T01:17:46.424 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.417+0000 7f499bf3b700 1 Processor -- start 2023-06-23T01:17:46.424 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.417+0000 7f499bf3b700 1 -- start start 2023-06-23T01:17:46.424 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.418+0000 7f499bf3b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4994070360 0x7f499410fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:17:46.424 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.418+0000 7f499bf3b700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49940719b0 0x7f499410ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:17:46.425 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.418+0000 7f499bf3b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4994104aa0 0x7f4994117040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:17:46.425 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.418+0000 7f499bf3b700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4994117650 con 0x7f49940719b0 2023-06-23T01:17:46.425 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.418+0000 7f499bf3b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49941177c0 con 0x7f4994104aa0 2023-06-23T01:17:46.425 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.418+0000 7f499bf3b700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4994117980 con 0x7f4994070360 2023-06-23T01:17:46.425 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.418+0000 7f4999cd7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4994070360 0x7f499410fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:17:46.426 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.418+0000 7f49994d6700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49940719b0 0x7f499410ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:17:46.426 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.418+0000 7f499a4d8700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4994104aa0 0x7f4994117040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:17:46.426 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.418+0000 7f4999cd7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4994070360 0x7f499410fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.120:45612/0 (socket says 172.21.15.120:45612) 2023-06-23T01:17:46.427 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.418+0000 7f49994d6700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49940719b0 0x7f499410ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35432/0 (socket says 172.21.15.120:35432) 2023-06-23T01:17:46.427 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.418+0000 7f4999cd7700 1 -- 172.21.15.120:0/4231717883 learned_addr learned my addr 172.21.15.120:0/4231717883 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:17:46.427 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.418+0000 7f49994d6700 1 -- 172.21.15.120:0/4231717883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4994070360 msgr2=0x7f499410fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:17:46.427 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.418+0000 7f49994d6700 1 --2- 172.21.15.120:0/4231717883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4994070360 0x7f499410fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:46.428 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.418+0000 7f49994d6700 1 -- 172.21.15.120:0/4231717883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4994104aa0 msgr2=0x7f4994117040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:17:46.428 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.418+0000 7f49994d6700 1 --2- 172.21.15.120:0/4231717883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4994104aa0 0x7f4994117040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:46.429 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.418+0000 7f49994d6700 1 -- 172.21.15.120:0/4231717883 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4990005040 con 0x7f49940719b0 2023-06-23T01:17:46.429 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.418+0000 7f4999cd7700 1 --2- 172.21.15.120:0/4231717883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4994070360 0x7f499410fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T01:17:46.429 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.419+0000 7f49994d6700 1 --2- 172.21.15.120:0/4231717883 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49940719b0 0x7f499410ffb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4984009fd0 tx=0x7f498400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:17:46.429 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.419+0000 7f498a7fc700 1 -- 172.21.15.120:0/4231717883 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4984007cc0 con 0x7f49940719b0 2023-06-23T01:17:46.430 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.419+0000 7f498a7fc700 1 -- 172.21.15.120:0/4231717883 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f498400f040 con 0x7f49940719b0 2023-06-23T01:17:46.430 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.419+0000 7f499bf3b700 1 -- 172.21.15.120:0/4231717883 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4994117c00 con 0x7f49940719b0 2023-06-23T01:17:46.430 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.419+0000 7f498a7fc700 1 -- 172.21.15.120:0/4231717883 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498400c980 con 0x7f49940719b0 2023-06-23T01:17:46.430 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.419+0000 7f499bf3b700 1 -- 172.21.15.120:0/4231717883 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49941180f0 con 0x7f49940719b0 2023-06-23T01:17:46.431 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.420+0000 7f499bf3b700 1 -- 172.21.15.120:0/4231717883 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49941031f0 con 0x7f49940719b0 2023-06-23T01:17:46.432 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.422+0000 7f498a7fc700 1 -- 172.21.15.120:0/4231717883 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7f49840090d0 con 0x7f49940719b0 2023-06-23T01:17:46.432 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.423+0000 7f498a7fc700 1 --2- 172.21.15.120:0/4231717883 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f498009b680 0x7f498009db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:17:46.433 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.423+0000 7f498a7fc700 1 -- 172.21.15.120:0/4231717883 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6843+0+0 (secure 0 0 0) 0x7f49840c44c0 con 0x7f49940719b0 2023-06-23T01:17:46.433 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.423+0000 7f4999cd7700 1 --2- 172.21.15.120:0/4231717883 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f498009b680 0x7f498009db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:17:46.433 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.424+0000 7f4999cd7700 1 --2- 172.21.15.120:0/4231717883 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f498009b680 0x7f498009db30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4990004890 tx=0x7f4990009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:17:46.433 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.426+0000 7f498a7fc700 1 -- 172.21.15.120:0/4231717883 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f4984090240 con 0x7f49940719b0 2023-06-23T01:17:46.619 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.616+0000 7f499bf3b700 1 -- 172.21.15.120:0/4231717883 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f499402b300 con 0x7f49940719b0 2023-06-23T01:17:46.620 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.617+0000 7f498a7fc700 1 -- 172.21.15.120:0/4231717883 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) v1 ==== 74+0+133 (secure 0 0 0) 0x7f49840190a0 con 0x7f49940719b0 2023-06-23T01:17:46.620 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:17:46.622 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.619+0000 7f499bf3b700 1 -- 172.21.15.120:0/4231717883 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f498009b680 msgr2=0x7f498009db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:17:46.622 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.619+0000 7f499bf3b700 1 --2- 172.21.15.120:0/4231717883 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f498009b680 0x7f498009db30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4990004890 tx=0x7f4990009e30 comp rx=0 tx=0).stop 2023-06-23T01:17:46.623 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.619+0000 7f499bf3b700 1 -- 172.21.15.120:0/4231717883 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49940719b0 msgr2=0x7f499410ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:17:46.623 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.619+0000 7f499bf3b700 1 --2- 172.21.15.120:0/4231717883 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49940719b0 0x7f499410ffb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4984009fd0 tx=0x7f498400b040 comp rx=0 tx=0).stop 2023-06-23T01:17:46.623 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.619+0000 7f499bf3b700 1 -- 172.21.15.120:0/4231717883 shutdown_connections 2023-06-23T01:17:46.623 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.619+0000 7f499bf3b700 1 --2- 172.21.15.120:0/4231717883 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f498009b680 0x7f498009db30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:46.623 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.619+0000 7f499bf3b700 1 --2- 172.21.15.120:0/4231717883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4994104aa0 0x7f4994117040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:46.623 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.619+0000 7f499bf3b700 1 --2- 172.21.15.120:0/4231717883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4994070360 0x7f499410fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:46.624 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.619+0000 7f499bf3b700 1 --2- 172.21.15.120:0/4231717883 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49940719b0 0x7f499410ffb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:46.624 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.619+0000 7f499bf3b700 1 -- 172.21.15.120:0/4231717883 >> 172.21.15.120:0/4231717883 conn(0x7f49940f8320 msgr2=0x7f49940fa700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:17:46.624 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.619+0000 7f499bf3b700 1 -- 172.21.15.120:0/4231717883 shutdown_connections 2023-06-23T01:17:46.624 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:46.619+0000 7f499bf3b700 1 -- 172.21.15.120:0/4231717883 wait complete. 2023-06-23T01:17:46.933 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:46 smithi120 ceph-mon[79813]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-06-23T01:17:46.933 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:46 smithi120 ceph-mon[79813]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2023-06-23T01:17:46.933 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:46 smithi120 ceph-mon[79813]: osdmap e79: 12 total, 11 up, 12 in 2023-06-23T01:17:46.933 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:46 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-23T01:17:46.934 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:46 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-23T01:17:46.934 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:46 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/4231717883' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-23T01:17:46.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:46 smithi134 ceph-mon[116146]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-06-23T01:17:46.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:46 smithi134 ceph-mon[116146]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2023-06-23T01:17:46.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:46 smithi134 ceph-mon[116146]: osdmap e79: 12 total, 11 up, 12 in 2023-06-23T01:17:46.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:46 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-23T01:17:46.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:46 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-23T01:17:46.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:46 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/4231717883' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-23T01:17:47.015 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":79,"num_osds":12,"num_up_osds":11,"osd_up_since":1687483041,"num_in_osds":12,"osd_in_since":1687483047,"num_remapped_pgs":0} 2023-06-23T01:17:47.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:46 smithi157 ceph-mon[116945]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-06-23T01:17:47.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:46 smithi157 ceph-mon[116945]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2023-06-23T01:17:47.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:46 smithi157 ceph-mon[116945]: osdmap e79: 12 total, 11 up, 12 in 2023-06-23T01:17:47.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:46 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-23T01:17:47.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:46 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-23T01:17:47.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:46 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/4231717883' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-23T01:17:47.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:47 smithi134 ceph-mon[116146]: purged_snaps scrub starts 2023-06-23T01:17:47.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:47 smithi134 ceph-mon[116146]: purged_snaps scrub ok 2023-06-23T01:17:47.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:47 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-23T01:17:47.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:47 smithi134 ceph-mon[116146]: osd.11 [v2:172.21.15.157:6824/1537477260,v1:172.21.15.157:6825/1537477260] boot 2023-06-23T01:17:47.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:47 smithi134 ceph-mon[116146]: osdmap e80: 12 total, 12 up, 12 in 2023-06-23T01:17:47.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:47 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-23T01:17:47.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:47 smithi134 ceph-mon[116146]: Detected new or changed devices on smithi157 2023-06-23T01:17:47.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:47 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:47.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:47 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:47.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:47 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:17:47.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:47 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:17:47.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:47 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:17:47.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:47 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:17:47.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:47 smithi134 ceph-mon[116146]: Adjusting osd_memory_target on smithi157 to 4293M 2023-06-23T01:17:47.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:47 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:47.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:47 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:17:47.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:47 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:17:47.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:47 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:48.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:47 smithi120 ceph-mon[79813]: purged_snaps scrub starts 2023-06-23T01:17:48.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:47 smithi120 ceph-mon[79813]: purged_snaps scrub ok 2023-06-23T01:17:48.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:47 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-23T01:17:48.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:47 smithi120 ceph-mon[79813]: osd.11 [v2:172.21.15.157:6824/1537477260,v1:172.21.15.157:6825/1537477260] boot 2023-06-23T01:17:48.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:47 smithi120 ceph-mon[79813]: osdmap e80: 12 total, 12 up, 12 in 2023-06-23T01:17:48.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:47 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-23T01:17:48.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:47 smithi120 ceph-mon[79813]: Detected new or changed devices on smithi157 2023-06-23T01:17:48.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:47 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:48.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:47 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:48.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:47 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:17:48.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:47 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:17:48.013 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:47 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:17:48.013 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:47 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:17:48.013 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:47 smithi120 ceph-mon[79813]: Adjusting osd_memory_target on smithi157 to 4293M 2023-06-23T01:17:48.013 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:47 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:48.014 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:47 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:17:48.014 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:47 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:17:48.014 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:47 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:48.016 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph osd stat -f json 2023-06-23T01:17:48.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:47 smithi157 ceph-mon[116945]: purged_snaps scrub starts 2023-06-23T01:17:48.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:47 smithi157 ceph-mon[116945]: purged_snaps scrub ok 2023-06-23T01:17:48.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:47 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-23T01:17:48.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:47 smithi157 ceph-mon[116945]: osd.11 [v2:172.21.15.157:6824/1537477260,v1:172.21.15.157:6825/1537477260] boot 2023-06-23T01:17:48.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:47 smithi157 ceph-mon[116945]: osdmap e80: 12 total, 12 up, 12 in 2023-06-23T01:17:48.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:47 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-23T01:17:48.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:47 smithi157 ceph-mon[116945]: Detected new or changed devices on smithi157 2023-06-23T01:17:48.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:47 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:48.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:47 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:48.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:47 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:17:48.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:47 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:17:48.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:47 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:17:48.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:47 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-06-23T01:17:48.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:47 smithi157 ceph-mon[116945]: Adjusting osd_memory_target on smithi157 to 4293M 2023-06-23T01:17:48.128 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:47 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:48.129 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:47 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:17:48.129 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:47 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:17:48.129 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:47 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:48.781 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T01:17:49.125 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:48 smithi120 ceph-mon[79813]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-06-23T01:17:49.126 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:48 smithi120 ceph-mon[79813]: osdmap e81: 12 total, 12 up, 12 in 2023-06-23T01:17:49.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:48 smithi157 ceph-mon[116945]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-06-23T01:17:49.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:48 smithi157 ceph-mon[116945]: osdmap e81: 12 total, 12 up, 12 in 2023-06-23T01:17:49.193 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:48 smithi134 ceph-mon[116146]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-06-23T01:17:49.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:48 smithi134 ceph-mon[116146]: osdmap e81: 12 total, 12 up, 12 in 2023-06-23T01:17:50.146 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.142+0000 7f52f2c21700 1 -- 172.21.15.120:0/1138157760 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52ec0fcc00 msgr2=0x7f52ec0fd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:17:50.146 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.142+0000 7f52f2c21700 1 --2- 172.21.15.120:0/1138157760 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52ec0fcc00 0x7f52ec0fd010 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f52e00049c0 tx=0x7f52e0009e30 comp rx=0 tx=0).stop 2023-06-23T01:17:50.146 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.142+0000 7f52f2c21700 1 -- 172.21.15.120:0/1138157760 shutdown_connections 2023-06-23T01:17:50.147 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.142+0000 7f52f2c21700 1 --2- 172.21.15.120:0/1138157760 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52ec0fcc00 0x7f52ec0fd010 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:50.147 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.142+0000 7f52f2c21700 1 --2- 172.21.15.120:0/1138157760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52ec0fe7a0 0x7f52ec105060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:50.147 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.142+0000 7f52f2c21700 1 --2- 172.21.15.120:0/1138157760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52ec0fde10 0x7f52ec0fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:50.147 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.142+0000 7f52f2c21700 1 -- 172.21.15.120:0/1138157760 >> 172.21.15.120:0/1138157760 conn(0x7f52ec0f8320 msgr2=0x7f52ec0fa790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:17:50.147 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.142+0000 7f52f2c21700 1 -- 172.21.15.120:0/1138157760 shutdown_connections 2023-06-23T01:17:50.147 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.142+0000 7f52f2c21700 1 -- 172.21.15.120:0/1138157760 wait complete. 2023-06-23T01:17:50.148 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.143+0000 7f52f2c21700 1 Processor -- start 2023-06-23T01:17:50.148 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.143+0000 7f52f2c21700 1 -- start start 2023-06-23T01:17:50.150 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.143+0000 7f52f2c21700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52ec0fcc00 0x7f52ec10d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:17:50.150 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.143+0000 7f52f2c21700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52ec0fde10 0x7f52ec10dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:17:50.150 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.143+0000 7f52f2c21700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52ec0fe7a0 0x7f52ec114e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:17:50.151 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.143+0000 7f52f2c21700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec115420 con 0x7f52ec0fcc00 2023-06-23T01:17:50.151 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.143+0000 7f52f2c21700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec115590 con 0x7f52ec0fde10 2023-06-23T01:17:50.151 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.143+0000 7f52f2c21700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec115750 con 0x7f52ec0fe7a0 2023-06-23T01:17:50.152 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.144+0000 7f52f09bd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52ec0fcc00 0x7f52ec10d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:17:50.152 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.144+0000 7f52f11be700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52ec0fe7a0 0x7f52ec114e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:17:50.152 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.144+0000 7f52f09bd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52ec0fcc00 0x7f52ec10d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35442/0 (socket says 172.21.15.120:35442) 2023-06-23T01:17:50.153 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.144+0000 7f52f09bd700 1 -- 172.21.15.120:0/3477844335 learned_addr learned my addr 172.21.15.120:0/3477844335 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:17:50.153 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.144+0000 7f52ebfff700 1 --2- 172.21.15.120:0/3477844335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52ec0fde10 0x7f52ec10dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:17:50.153 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.144+0000 7f52f09bd700 1 -- 172.21.15.120:0/3477844335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52ec0fe7a0 msgr2=0x7f52ec114e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:17:50.153 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.144+0000 7f52f09bd700 1 --2- 172.21.15.120:0/3477844335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52ec0fe7a0 0x7f52ec114e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:50.154 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.144+0000 7f52f09bd700 1 -- 172.21.15.120:0/3477844335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52ec0fde10 msgr2=0x7f52ec10dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:17:50.154 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.144+0000 7f52f09bd700 1 --2- 172.21.15.120:0/3477844335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52ec0fde10 0x7f52ec10dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:50.154 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.144+0000 7f52f09bd700 1 -- 172.21.15.120:0/3477844335 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e0005040 con 0x7f52ec0fcc00 2023-06-23T01:17:50.155 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.144+0000 7f52f09bd700 1 --2- 172.21.15.120:0/3477844335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52ec0fcc00 0x7f52ec10d840 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f52e000b3b0 tx=0x7f52e0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:17:50.155 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.144+0000 7f52e97fa700 1 -- 172.21.15.120:0/3477844335 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e000e040 con 0x7f52ec0fcc00 2023-06-23T01:17:50.155 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.145+0000 7f52f2c21700 1 -- 172.21.15.120:0/3477844335 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52ec1159d0 con 0x7f52ec0fcc00 2023-06-23T01:17:50.155 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.145+0000 7f52e97fa700 1 -- 172.21.15.120:0/3477844335 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52e0013420 con 0x7f52ec0fcc00 2023-06-23T01:17:50.156 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.145+0000 7f52e97fa700 1 -- 172.21.15.120:0/3477844335 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e001c680 con 0x7f52ec0fcc00 2023-06-23T01:17:50.156 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.145+0000 7f52f2c21700 1 -- 172.21.15.120:0/3477844335 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52ec115e30 con 0x7f52ec0fcc00 2023-06-23T01:17:50.156 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.146+0000 7f52f2c21700 1 -- 172.21.15.120:0/3477844335 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52ec103ac0 con 0x7f52ec0fcc00 2023-06-23T01:17:50.158 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.147+0000 7f52e97fa700 1 -- 172.21.15.120:0/3477844335 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7f52e0013590 con 0x7f52ec0fcc00 2023-06-23T01:17:50.158 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.149+0000 7f52e97fa700 1 --2- 172.21.15.120:0/3477844335 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f52dc09b680 0x7f52dc09db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:17:50.158 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.149+0000 7f52e97fa700 1 -- 172.21.15.120:0/3477844335 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f52e00c5cd0 con 0x7f52ec0fcc00 2023-06-23T01:17:50.159 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.149+0000 7f52ebfff700 1 --2- 172.21.15.120:0/3477844335 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f52dc09b680 0x7f52dc09db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:17:50.159 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.149+0000 7f52ebfff700 1 --2- 172.21.15.120:0/3477844335 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f52dc09b680 0x7f52dc09db30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f52d8009b80 tx=0x7f52d800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:17:50.159 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.151+0000 7f52e97fa700 1 -- 172.21.15.120:0/3477844335 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f52e0091930 con 0x7f52ec0fcc00 2023-06-23T01:17:50.349 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.346+0000 7f52f2c21700 1 -- 172.21.15.120:0/3477844335 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f52ec02b0e0 con 0x7f52ec0fcc00 2023-06-23T01:17:50.350 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.347+0000 7f52e97fa700 1 -- 172.21.15.120:0/3477844335 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) v1 ==== 74+0+133 (secure 0 0 0) 0x7f52e0094f50 con 0x7f52ec0fcc00 2023-06-23T01:17:50.350 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:17:50.352 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.349+0000 7f52f2c21700 1 -- 172.21.15.120:0/3477844335 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f52dc09b680 msgr2=0x7f52dc09db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:17:50.352 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.349+0000 7f52f2c21700 1 --2- 172.21.15.120:0/3477844335 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f52dc09b680 0x7f52dc09db30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f52d8009b80 tx=0x7f52d800b040 comp rx=0 tx=0).stop 2023-06-23T01:17:50.352 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.349+0000 7f52f2c21700 1 -- 172.21.15.120:0/3477844335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52ec0fcc00 msgr2=0x7f52ec10d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:17:50.352 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.349+0000 7f52f2c21700 1 --2- 172.21.15.120:0/3477844335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52ec0fcc00 0x7f52ec10d840 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f52e000b3b0 tx=0x7f52e0009e30 comp rx=0 tx=0).stop 2023-06-23T01:17:50.353 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.349+0000 7f52f2c21700 1 -- 172.21.15.120:0/3477844335 shutdown_connections 2023-06-23T01:17:50.353 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.349+0000 7f52f2c21700 1 --2- 172.21.15.120:0/3477844335 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f52dc09b680 0x7f52dc09db30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:50.353 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.349+0000 7f52f2c21700 1 --2- 172.21.15.120:0/3477844335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52ec0fcc00 0x7f52ec10d840 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:50.353 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.349+0000 7f52f2c21700 1 --2- 172.21.15.120:0/3477844335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52ec0fe7a0 0x7f52ec114e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:50.353 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.349+0000 7f52f2c21700 1 --2- 172.21.15.120:0/3477844335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52ec0fde10 0x7f52ec10dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:50.353 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.349+0000 7f52f2c21700 1 -- 172.21.15.120:0/3477844335 >> 172.21.15.120:0/3477844335 conn(0x7f52ec0f8320 msgr2=0x7f52ec0fa5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:17:50.353 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.349+0000 7f52f2c21700 1 -- 172.21.15.120:0/3477844335 shutdown_connections 2023-06-23T01:17:50.354 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:50.349+0000 7f52f2c21700 1 -- 172.21.15.120:0/3477844335 wait complete. 2023-06-23T01:17:50.741 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1687483066,"num_in_osds":12,"osd_in_since":1687483047,"num_remapped_pgs":0} 2023-06-23T01:17:50.742 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph osd dump --format=json 2023-06-23T01:17:51.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:50 smithi120 ceph-mon[79813]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:17:51.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:50 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/3477844335' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-23T01:17:51.126 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:50 smithi157 ceph-mon[116945]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:17:51.127 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:50 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/3477844335' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-23T01:17:51.193 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:50 smithi134 ceph-mon[116146]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:17:51.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:50 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/3477844335' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-23T01:17:51.502 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T01:17:52.871 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.868+0000 7f1ee568b700 1 -- 172.21.15.120:0/3243153901 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ee00fe600 msgr2=0x7f1ee0106c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:17:52.872 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.868+0000 7f1ee568b700 1 --2- 172.21.15.120:0/3243153901 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ee00fe600 0x7f1ee0106c40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1ed40049c0 tx=0x7f1ed4009e30 comp rx=0 tx=0).stop 2023-06-23T01:17:52.872 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.868+0000 7f1ee568b700 1 -- 172.21.15.120:0/3243153901 shutdown_connections 2023-06-23T01:17:52.872 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.868+0000 7f1ee568b700 1 --2- 172.21.15.120:0/3243153901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ee0107210 0x7f1ee01095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:52.872 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.868+0000 7f1ee568b700 1 --2- 172.21.15.120:0/3243153901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ee00fdc60 0x7f1ee00fe030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:52.872 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.868+0000 7f1ee568b700 1 --2- 172.21.15.120:0/3243153901 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ee00fe600 0x7f1ee0106c40 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:52.873 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.868+0000 7f1ee568b700 1 -- 172.21.15.120:0/3243153901 >> 172.21.15.120:0/3243153901 conn(0x7f1ee0072c30 msgr2=0x7f1ee0073030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:17:52.873 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.868+0000 7f1ee568b700 1 -- 172.21.15.120:0/3243153901 shutdown_connections 2023-06-23T01:17:52.873 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.868+0000 7f1ee568b700 1 -- 172.21.15.120:0/3243153901 wait complete. 2023-06-23T01:17:52.876 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.869+0000 7f1ee568b700 1 Processor -- start 2023-06-23T01:17:52.876 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.869+0000 7f1ee568b700 1 -- start start 2023-06-23T01:17:52.876 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.869+0000 7f1ee568b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ee00fdc60 0x7f1ee00fba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:17:52.877 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.869+0000 7f1ee568b700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ee00fe600 0x7f1ee00fbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:17:52.877 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.869+0000 7f1ee568b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ee0107210 0x7f1ee00fc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:17:52.877 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.869+0000 7f1ee568b700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee011bfa0 con 0x7f1ee00fe600 2023-06-23T01:17:52.877 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.869+0000 7f1ee568b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee011c110 con 0x7f1ee00fdc60 2023-06-23T01:17:52.878 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.869+0000 7f1ee568b700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee011c2d0 con 0x7f1ee0107210 2023-06-23T01:17:52.878 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.869+0000 7f1ede7fc700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ee00fe600 0x7f1ee00fbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:17:52.878 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.870+0000 7f1ede7fc700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ee00fe600 0x7f1ee00fbfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50126/0 (socket says 172.21.15.120:50126) 2023-06-23T01:17:52.879 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.870+0000 7f1ede7fc700 1 -- 172.21.15.120:0/2204108354 learned_addr learned my addr 172.21.15.120:0/2204108354 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:17:52.879 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.870+0000 7f1edeffd700 1 --2- 172.21.15.120:0/2204108354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ee00fdc60 0x7f1ee00fba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:17:52.879 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.870+0000 7f1edf7fe700 1 --2- 172.21.15.120:0/2204108354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ee0107210 0x7f1ee00fc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:17:52.879 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.870+0000 7f1ede7fc700 1 -- 172.21.15.120:0/2204108354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ee0107210 msgr2=0x7f1ee00fc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:17:52.880 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.870+0000 7f1ede7fc700 1 --2- 172.21.15.120:0/2204108354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ee0107210 0x7f1ee00fc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:52.880 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.870+0000 7f1ede7fc700 1 -- 172.21.15.120:0/2204108354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ee00fdc60 msgr2=0x7f1ee00fba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:17:52.880 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.870+0000 7f1ede7fc700 1 --2- 172.21.15.120:0/2204108354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ee00fdc60 0x7f1ee00fba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:52.880 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.870+0000 7f1ede7fc700 1 -- 172.21.15.120:0/2204108354 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed4005040 con 0x7f1ee00fe600 2023-06-23T01:17:52.881 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.870+0000 7f1ede7fc700 1 --2- 172.21.15.120:0/2204108354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ee00fe600 0x7f1ee00fbfd0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4009c90 tx=0x7f1ed4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:17:52.881 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.870+0000 7f1ec77fe700 1 -- 172.21.15.120:0/2204108354 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed400e040 con 0x7f1ee00fe600 2023-06-23T01:17:52.881 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.870+0000 7f1ec77fe700 1 -- 172.21.15.120:0/2204108354 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ed40096f0 con 0x7f1ee00fe600 2023-06-23T01:17:52.882 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.870+0000 7f1ec77fe700 1 -- 172.21.15.120:0/2204108354 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed4013d00 con 0x7f1ee00fe600 2023-06-23T01:17:52.882 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.870+0000 7f1ee568b700 1 -- 172.21.15.120:0/2204108354 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ee011c550 con 0x7f1ee00fe600 2023-06-23T01:17:52.882 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.870+0000 7f1ee568b700 1 -- 172.21.15.120:0/2204108354 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ee011c960 con 0x7f1ee00fe600 2023-06-23T01:17:52.882 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.872+0000 7f1ee568b700 1 -- 172.21.15.120:0/2204108354 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ee004c1b0 con 0x7f1ee00fe600 2023-06-23T01:17:52.883 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.873+0000 7f1ec77fe700 1 -- 172.21.15.120:0/2204108354 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7f1ed4013420 con 0x7f1ee00fe600 2023-06-23T01:17:52.883 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.875+0000 7f1ec77fe700 1 --2- 172.21.15.120:0/2204108354 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f1ec809b700 0x7f1ec809dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:17:52.883 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.876+0000 7f1ec77fe700 1 -- 172.21.15.120:0/2204108354 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f1ed40c5b10 con 0x7f1ee00fe600 2023-06-23T01:17:52.883 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.876+0000 7f1edeffd700 1 --2- 172.21.15.120:0/2204108354 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f1ec809b700 0x7f1ec809dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:17:52.884 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.876+0000 7f1edeffd700 1 --2- 172.21.15.120:0/2204108354 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f1ec809b700 0x7f1ec809dbb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0004f40 tx=0x7f1ed000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:17:52.884 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:52.880+0000 7f1ec77fe700 1 -- 172.21.15.120:0/2204108354 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f1ed4091770 con 0x7f1ee00fe600 2023-06-23T01:17:53.077 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:53.074+0000 7f1ee568b700 1 -- 172.21.15.120:0/2204108354 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ee004c1b0 con 0x7f1ee00fe600 2023-06-23T01:17:53.078 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:53.075+0000 7f1ec77fe700 1 -- 172.21.15.120:0/2204108354 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18241 (secure 0 0 0) 0x7f1ed4013e60 con 0x7f1ee00fe600 2023-06-23T01:17:53.079 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:17:53.079 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":81,"fsid":"9efbfa8e-1162-11ee-9b2b-001a4aab830c","created":"2023-06-23T01:10:46.840474+0000","modified":"2023-06-23T01:17:47.828105+0000","last_up_change":"2023-06-23T01:17:46.670887+0000","last_in_change":"2023-06-23T01:17:27.552330+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T01:14:03.589464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"3729642b-e74f-4253-bde8-535d8bfe8c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3479894866},{"type":"v1","addr":"172.21.15.120:6803","nonce":3479894866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3479894866},{"type":"v1","addr":"172.21.15.120:6805","nonce":3479894866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3479894866},{"type":"v1","addr":"172.21.15.120:6809","nonce":3479894866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3479894866},{"type":"v1","addr":"172.21.15.120:6807","nonce":3479894866}]},"public_addr":"172.21.15.120:6803/3479894866","cluster_addr":"172.21.15.120:6805/3479894866","heartbeat_back_addr":"172.21.15.120:6809/3479894866","heartbeat_front_addr":"172.21.15.120:6807/3479894866","state":["exists","up"]},{"osd":1,"uuid":"58240258-afd5-4991-9d5b-a80d365e4fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3046662876},{"type":"v1","addr":"172.21.15.134:6801","nonce":3046662876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3046662876},{"type":"v1","addr":"172.21.15.134:6803","nonce":3046662876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3046662876},{"type":"v1","addr":"172.21.15.134:6807","nonce":3046662876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3046662876},{"type":"v1","addr":"172.21.15.134:6805","nonce":3046662876}]},"public_addr":"172.21.15.134:6801/3046662876","cluster_addr":"172.21.15.134:6803/3046662876","heartbeat_back_addr":"172.21.15.134:6807/3046662876","heartbeat_front_addr":"172.21.15.134:6805/3046662876","state":["exists","up"]},{"osd":2,"uuid":"05988a7a-7cf2-4fbb-9d4f-22604cd2f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3996265223},{"type":"v1","addr":"172.21.15.157:6801","nonce":3996265223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3996265223},{"type":"v1","addr":"172.21.15.157:6803","nonce":3996265223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3996265223},{"type":"v1","addr":"172.21.15.157:6807","nonce":3996265223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3996265223},{"type":"v1","addr":"172.21.15.157:6805","nonce":3996265223}]},"public_addr":"172.21.15.157:6801/3996265223","cluster_addr":"172.21.15.157:6803/3996265223","heartbeat_back_addr":"172.21.15.157:6807/3996265223","heartbeat_front_addr":"172.21.15.157:6805/3996265223","state":["exists","up"]},{"osd":3,"uuid":"7d9e9739-f043-43d6-8e46-7711dcf43e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":1716638437},{"type":"v1","addr":"172.21.15.120:6811","nonce":1716638437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":1716638437},{"type":"v1","addr":"172.21.15.120:6813","nonce":1716638437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1716638437},{"type":"v1","addr":"172.21.15.120:6817","nonce":1716638437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":1716638437},{"type":"v1","addr":"172.21.15.120:6815","nonce":1716638437}]},"public_addr":"172.21.15.120:6811/1716638437","cluster_addr":"172.21.15.120:6813/1716638437","heartbeat_back_addr":"172.21.15.120:6817/1716638437","heartbeat_front_addr":"172.21.15.120:6815/1716638437","state":["exists","up"]},{"osd":4,"uuid":"3a70f86d-f903-4618-be66-9aa1eac44ccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4172238262},{"type":"v1","addr":"172.21.15.134:6809","nonce":4172238262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4172238262},{"type":"v1","addr":"172.21.15.134:6811","nonce":4172238262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4172238262},{"type":"v1","addr":"172.21.15.134:6815","nonce":4172238262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4172238262},{"type":"v1","addr":"172.21.15.134:6813","nonce":4172238262}]},"public_addr":"172.21.15.134:6809/4172238262","cluster_addr":"172.21.15.134:6811/4172238262","heartbeat_back_addr":"172.21.15.134:6815/4172238262","heartbeat_front_addr":"172.21.15.134:6813/4172238262","state":["exists","up"]},{"osd":5,"uuid":"b2316488-75c3-4847-b4c4-998730f27c88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3374822045},{"type":"v1","addr":"172.21.15.157:6809","nonce":3374822045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3374822045},{"type":"v1","addr":"172.21.15.157:6811","nonce":3374822045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3374822045},{"type":"v1","addr":"172.21.15.157:6815","nonce":3374822045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3374822045},{"type":"v1","addr":"172.21.15.157:6813","nonce":3374822045}]},"public_addr":"172.21.15.157:6809/3374822045","cluster_addr":"172.21.15.157:6811/3374822045","heartbeat_back_addr":"172.21.15.157:6815/3374822045","heartbeat_front_addr":"172.21.15.157:6813/3374822045","state":["exists","up"]},{"osd":6,"uuid":"338c2c4a-d982-4031-aaaf-be76c9dd74a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2097345225},{"type":"v1","addr":"172.21.15.120:6819","nonce":2097345225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2097345225},{"type":"v1","addr":"172.21.15.120:6821","nonce":2097345225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2097345225},{"type":"v1","addr":"172.21.15.120:6825","nonce":2097345225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2097345225},{"type":"v1","addr":"172.21.15.120:6823","nonce":2097345225}]},"public_addr":"172.21.15.120:6819/2097345225","cluster_addr":"172.21.15.120:6821/2097345225","heartbeat_back_addr":"172.21.15.120:6825/2097345225","heartbeat_front_addr":"172.21.15.120:6823/2097345225","state":["exists","up"]},{"osd":7,"uuid":"bff92799-ae9c-4409-be65-7ffc27a1c3f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1345745298},{"type":"v1","addr":"172.21.15.134:6817","nonce":1345745298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1345745298},{"type":"v1","addr":"172.21.15.134:6819","nonce":1345745298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1345745298},{"type":"v1","addr":"172.21.15.134:6823","nonce":1345745298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1345745298},{"type":"v1","addr":"172.21.15.134:6821","nonce":1345745298}]},"public_addr":"172.21.15.134:6817/1345745298","cluster_addr":"172.21.15.134:6819/1345745298","heartbeat_back_addr":"172.21.15.134:6823/1345745298","heartbeat_front_addr":"172.21.15.134:6821/1345745298","state":["exists","up"]},{"osd":8,"uuid":"e4aac47e-d7f9-4feb-ada7-d8103fdca246","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1124549599},{"type":"v1","addr":"172.21.15.157:6817","nonce":1124549599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1124549599},{"type":"v1","addr":"172.21.15.157:6819","nonce":1124549599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1124549599},{"type":"v1","addr":"172.21.15.157:6823","nonce":1124549599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1124549599},{"type":"v1","addr":"172.21.15.157:6821","nonce":1124549599}]},"public_addr":"172.21.15.157:6817/1124549599","cluster_addr":"172.21.15.157:6819/1124549599","heartbeat_back_addr":"172.21.15.157:6823/1124549599","heartbeat_front_addr":"172.21.15.157:6821/1124549599","state":["exists","up"]},{"osd":9,"uuid":"48b988ba-e866-47d1-aca9-09f6ad815bd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1402414981},{"type":"v1","addr":"172.21.15.120:6827","nonce":1402414981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1402414981},{"type":"v1","addr":"172.21.15.120:6829","nonce":1402414981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1402414981},{"type":"v1","addr":"172.21.15.120:6833","nonce":1402414981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1402414981},{"type":"v1","addr":"172.21.15.120:6831","nonce":1402414981}]},"public_addr":"172.21.15.120:6827/1402414981","cluster_addr":"172.21.15.120:6829/1402414981","heartbeat_back_addr":"172.21.15.120:6833/1402414981","heartbeat_front_addr":"172.21.15.120:6831/1402414981","state":["exists","up"]},{"osd":10,"uuid":"777f3add-3779-4221-a86f-65cca2e5e8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":4041441144},{"type":"v1","addr":"172.21.15.134:6825","nonce":4041441144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":4041441144},{"type":"v1","addr":"172.21.15.134:6827","nonce":4041441144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":4041441144},{"type":"v1","addr":"172.21.15.134:6831","nonce":4041441144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":4041441144},{"type":"v1","addr":"172.21.15.134:6829","nonce":4041441144}]},"public_addr":"172.21.15.134:6825/4041441144","cluster_addr":"172.21.15.134:6827/4041441144","heartbeat_back_addr":"172.21.15.134:6831/4041441144","heartbeat_front_addr":"172.21.15.134:6829/4041441144","state":["exists","up"]},{"osd":11,"uuid":"7166e11b-03c0-45bf-abbf-a74c4a13a39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1537477260},{"type":"v1","addr":"172.21.15.157:6825","nonce":1537477260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1537477260},{"type":"v1","addr":"172.21.15.157:6827","nonce":1537477260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1537477260},{"type":"v1","addr":"172.21.15.157:6831","nonce":1537477260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1537477260},{"type":"v1","addr":"172.21.15.157:6829","nonce":1537477260}]},"public_addr":"172.21.15.157:6825/1537477260","cluster_addr":"172.21.15.157:6827/1537477260","heartbeat_back_addr":"172.21.15.157:6831/1537477260","heartbeat_front_addr":"172.21.15.157:6829/1537477260","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:13:13.580631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:13:37.472924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:14:01.591555+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:14:28.472130+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:14:52.356433+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:15:15.946523+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:15:42.036105+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:16:06.154934+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:16:30.043788+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:16:55.975403+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:17:20.612778+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:17:44.896301+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.120:0/2865552846":"2023-06-24T01:11:38.967204+0000","172.21.15.120:0/2163781512":"2023-06-24T01:11:38.967204+0000","172.21.15.120:0/1943345772":"2023-06-24T01:11:38.967204+0000","172.21.15.120:6801/1836713308":"2023-06-24T01:11:38.967204+0000","172.21.15.120:0/2358086231":"2023-06-24T01:11:14.574788+0000","172.21.15.120:6801/506843621":"2023-06-24T01:11:14.574788+0000","172.21.15.120:6800/506843621":"2023-06-24T01:11:14.574788+0000","172.21.15.120:0/4077038498":"2023-06-24T01:11:14.574788+0000","172.21.15.120:6800/1836713308":"2023-06-24T01:11:38.967204+0000","172.21.15.120:0/2484415475":"2023-06-24T01:11:14.574788+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T01:17:53.081 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:53.078+0000 7f1ee568b700 1 -- 172.21.15.120:0/2204108354 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f1ec809b700 msgr2=0x7f1ec809dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:17:53.081 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:53.078+0000 7f1ee568b700 1 --2- 172.21.15.120:0/2204108354 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f1ec809b700 0x7f1ec809dbb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0004f40 tx=0x7f1ed000b040 comp rx=0 tx=0).stop 2023-06-23T01:17:53.081 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:53.078+0000 7f1ee568b700 1 -- 172.21.15.120:0/2204108354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ee00fe600 msgr2=0x7f1ee00fbfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:17:53.081 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:53.078+0000 7f1ee568b700 1 --2- 172.21.15.120:0/2204108354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ee00fe600 0x7f1ee00fbfd0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4009c90 tx=0x7f1ed4009e30 comp rx=0 tx=0).stop 2023-06-23T01:17:53.081 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:53.078+0000 7f1ee568b700 1 -- 172.21.15.120:0/2204108354 shutdown_connections 2023-06-23T01:17:53.082 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:53.078+0000 7f1ee568b700 1 --2- 172.21.15.120:0/2204108354 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f1ec809b700 0x7f1ec809dbb0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:53.082 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:53.078+0000 7f1ee568b700 1 --2- 172.21.15.120:0/2204108354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ee0107210 0x7f1ee00fc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:53.082 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:53.078+0000 7f1ee568b700 1 --2- 172.21.15.120:0/2204108354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ee00fdc60 0x7f1ee00fba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:53.082 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:53.078+0000 7f1ee568b700 1 --2- 172.21.15.120:0/2204108354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ee00fe600 0x7f1ee00fbfd0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:53.082 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:53.078+0000 7f1ee568b700 1 -- 172.21.15.120:0/2204108354 >> 172.21.15.120:0/2204108354 conn(0x7f1ee0072c30 msgr2=0x7f1ee00febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:17:53.082 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:53.078+0000 7f1ee568b700 1 -- 172.21.15.120:0/2204108354 shutdown_connections 2023-06-23T01:17:53.082 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:53.078+0000 7f1ee568b700 1 -- 172.21.15.120:0/2204108354 wait complete. 2023-06-23T01:17:53.236 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:52 smithi120 ceph-mon[79813]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:17:53.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:52 smithi157 ceph-mon[116945]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:17:53.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:52 smithi134 ceph-mon[116146]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:17:53.463 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-23T01:14:03.589464+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-06-23T01:17:53.464 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph osd pool get .mgr pg_num 2023-06-23T01:17:54.219 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T01:17:54.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:53 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/2204108354' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T01:17:54.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:53 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/2204108354' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T01:17:54.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:53 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/2204108354' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T01:17:55.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:54 smithi120 ceph-mon[79813]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:17:55.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:54 smithi157 ceph-mon[116945]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:17:55.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:54 smithi134 ceph-mon[116146]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:17:55.597 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.594+0000 7f8189120700 1 -- 172.21.15.120:0/3552389072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f81840ff520 msgr2=0x7f818410bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:17:55.598 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.594+0000 7f8189120700 1 --2- 172.21.15.120:0/3552389072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f81840ff520 0x7f818410bc10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f817c009d90 tx=0x7f817c004d10 comp rx=0 tx=0).stop 2023-06-23T01:17:55.598 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.594+0000 7f8189120700 1 -- 172.21.15.120:0/3552389072 shutdown_connections 2023-06-23T01:17:55.598 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.594+0000 7f8189120700 1 --2- 172.21.15.120:0/3552389072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f81840ff520 0x7f818410bc10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:55.598 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.594+0000 7f8189120700 1 --2- 172.21.15.120:0/3552389072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8184104e30 0x7f8184105200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:55.598 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.594+0000 7f8189120700 1 --2- 172.21.15.120:0/3552389072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81841057d0 0x7f81840fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:55.599 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.594+0000 7f8189120700 1 -- 172.21.15.120:0/3552389072 >> 172.21.15.120:0/3552389072 conn(0x7f81840fa960 msgr2=0x7f81840fcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:17:55.599 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.594+0000 7f8189120700 1 -- 172.21.15.120:0/3552389072 shutdown_connections 2023-06-23T01:17:55.599 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.594+0000 7f8189120700 1 -- 172.21.15.120:0/3552389072 wait complete. 2023-06-23T01:17:55.600 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.595+0000 7f8189120700 1 Processor -- start 2023-06-23T01:17:55.601 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.595+0000 7f8189120700 1 -- start start 2023-06-23T01:17:55.601 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.595+0000 7f8189120700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81840ff520 0x7f81841158f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:17:55.601 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.595+0000 7f8189120700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8184104e30 0x7f8184115e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:17:55.602 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.595+0000 7f8189120700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81841057d0 0x7f818410f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:17:55.602 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.595+0000 7f8189120700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8184110030 con 0x7f8184104e30 2023-06-23T01:17:55.602 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.595+0000 7f8189120700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81841101a0 con 0x7f81841057d0 2023-06-23T01:17:55.602 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.595+0000 7f8189120700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8184110360 con 0x7f81840ff520 2023-06-23T01:17:55.602 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.596+0000 7f818259c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8184104e30 0x7f8184115e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:17:55.602 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.596+0000 7f818259c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8184104e30 0x7f8184115e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50154/0 (socket says 172.21.15.120:50154) 2023-06-23T01:17:55.602 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.596+0000 7f818259c700 1 -- 172.21.15.120:0/619020327 learned_addr learned my addr 172.21.15.120:0/619020327 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:17:55.603 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.596+0000 7f8182d9d700 1 --2- 172.21.15.120:0/619020327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81840ff520 0x7f81841158f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:17:55.603 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.596+0000 7f818359e700 1 --2- 172.21.15.120:0/619020327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81841057d0 0x7f818410f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:17:55.603 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.596+0000 7f818259c700 1 -- 172.21.15.120:0/619020327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81840ff520 msgr2=0x7f81841158f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:17:55.603 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.596+0000 7f818259c700 1 --2- 172.21.15.120:0/619020327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81840ff520 0x7f81841158f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:55.603 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.596+0000 7f818259c700 1 -- 172.21.15.120:0/619020327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81841057d0 msgr2=0x7f818410f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:17:55.603 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.596+0000 7f818259c700 1 --2- 172.21.15.120:0/619020327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81841057d0 0x7f818410f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:55.603 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.596+0000 7f818259c700 1 -- 172.21.15.120:0/619020327 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f817c009940 con 0x7f8184104e30 2023-06-23T01:17:55.604 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.596+0000 7f818259c700 1 --2- 172.21.15.120:0/619020327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8184104e30 0x7f8184115e30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f817800ebc0 tx=0x7f817800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:17:55.604 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.597+0000 7f816b7fe700 1 -- 172.21.15.120:0/619020327 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8178007cc0 con 0x7f8184104e30 2023-06-23T01:17:55.604 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.597+0000 7f816b7fe700 1 -- 172.21.15.120:0/619020327 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f817800f040 con 0x7f8184104e30 2023-06-23T01:17:55.604 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.597+0000 7f8189120700 1 -- 172.21.15.120:0/619020327 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81841105e0 con 0x7f8184104e30 2023-06-23T01:17:55.604 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.597+0000 7f816b7fe700 1 -- 172.21.15.120:0/619020327 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817800c990 con 0x7f8184104e30 2023-06-23T01:17:55.604 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.597+0000 7f8189120700 1 -- 172.21.15.120:0/619020327 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f818411c8b0 con 0x7f8184104e30 2023-06-23T01:17:55.604 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.598+0000 7f8189120700 1 -- 172.21.15.120:0/619020327 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f818404c1b0 con 0x7f8184104e30 2023-06-23T01:17:55.605 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.599+0000 7f816b7fe700 1 -- 172.21.15.120:0/619020327 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7f817800cd40 con 0x7f8184104e30 2023-06-23T01:17:55.605 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.601+0000 7f816b7fe700 1 --2- 172.21.15.120:0/619020327 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f816c09b680 0x7f816c09db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:17:55.606 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.601+0000 7f816b7fe700 1 -- 172.21.15.120:0/619020327 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f81780c44b0 con 0x7f8184104e30 2023-06-23T01:17:55.606 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.601+0000 7f8182d9d700 1 --2- 172.21.15.120:0/619020327 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f816c09b680 0x7f816c09db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:17:55.606 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.601+0000 7f8182d9d700 1 --2- 172.21.15.120:0/619020327 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f816c09b680 0x7f816c09db30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8174009b80 tx=0x7f817400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:17:55.607 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.604+0000 7f816b7fe700 1 -- 172.21.15.120:0/619020327 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f817804da00 con 0x7f8184104e30 2023-06-23T01:17:55.801 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.798+0000 7f8189120700 1 -- 172.21.15.120:0/619020327 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f818411caa0 con 0x7f8184104e30 2023-06-23T01:17:55.802 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.799+0000 7f816b7fe700 1 -- 172.21.15.120:0/619020327 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8178090110 con 0x7f8184104e30 2023-06-23T01:17:55.802 INFO:teuthology.orchestra.run.smithi120.stdout:pg_num: 1 2023-06-23T01:17:55.804 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.801+0000 7f8189120700 1 -- 172.21.15.120:0/619020327 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f816c09b680 msgr2=0x7f816c09db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:17:55.804 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.801+0000 7f8189120700 1 --2- 172.21.15.120:0/619020327 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f816c09b680 0x7f816c09db30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8174009b80 tx=0x7f817400b040 comp rx=0 tx=0).stop 2023-06-23T01:17:55.804 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.801+0000 7f8189120700 1 -- 172.21.15.120:0/619020327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8184104e30 msgr2=0x7f8184115e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:17:55.805 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.801+0000 7f8189120700 1 --2- 172.21.15.120:0/619020327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8184104e30 0x7f8184115e30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f817800ebc0 tx=0x7f817800b040 comp rx=0 tx=0).stop 2023-06-23T01:17:55.805 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.801+0000 7f8189120700 1 -- 172.21.15.120:0/619020327 shutdown_connections 2023-06-23T01:17:55.805 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.801+0000 7f8189120700 1 --2- 172.21.15.120:0/619020327 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f816c09b680 0x7f816c09db30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:55.805 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.801+0000 7f8189120700 1 --2- 172.21.15.120:0/619020327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81841057d0 0x7f818410f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:55.805 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.801+0000 7f8189120700 1 --2- 172.21.15.120:0/619020327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81840ff520 0x7f81841158f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:55.805 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.801+0000 7f8189120700 1 --2- 172.21.15.120:0/619020327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8184104e30 0x7f8184115e30 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:55.805 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.801+0000 7f8189120700 1 -- 172.21.15.120:0/619020327 >> 172.21.15.120:0/619020327 conn(0x7f81840fa960 msgr2=0x7f81841029f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:17:55.806 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.801+0000 7f8189120700 1 -- 172.21.15.120:0/619020327 shutdown_connections 2023-06-23T01:17:55.806 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:55.801+0000 7f8189120700 1 -- 172.21.15.120:0/619020327 wait complete. 2023-06-23T01:17:56.114 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:55 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/619020327' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T01:17:56.179 INFO:tasks.cephadm:Adding mds.a on smithi120 2023-06-23T01:17:56.179 INFO:tasks.cephadm:Adding mds.d on smithi120 2023-06-23T01:17:56.180 INFO:tasks.cephadm:Adding mds.g on smithi120 2023-06-23T01:17:56.180 INFO:tasks.cephadm:Adding mds.j on smithi120 2023-06-23T01:17:56.180 INFO:tasks.cephadm:Adding mds.b on smithi134 2023-06-23T01:17:56.180 INFO:tasks.cephadm:Adding mds.e on smithi134 2023-06-23T01:17:56.180 INFO:tasks.cephadm:Adding mds.h on smithi134 2023-06-23T01:17:56.181 INFO:tasks.cephadm:Adding mds.k on smithi134 2023-06-23T01:17:56.181 INFO:tasks.cephadm:Adding mds.c on smithi157 2023-06-23T01:17:56.181 INFO:tasks.cephadm:Adding mds.f on smithi157 2023-06-23T01:17:56.181 INFO:tasks.cephadm:Adding mds.i on smithi157 2023-06-23T01:17:56.181 INFO:tasks.cephadm:Adding mds.l on smithi157 2023-06-23T01:17:56.182 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c -- ceph orch apply mds all '12;smithi120=a;smithi120=d;smithi120=g;smithi120=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l' 2023-06-23T01:17:56.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:55 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/619020327' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T01:17:56.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:55 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/619020327' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T01:17:56.775 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/mon.c/config 2023-06-23T01:17:57.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:56 smithi120 ceph-mon[79813]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:17:57.322 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:56 smithi157 ceph-mon[116945]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:17:57.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:56 smithi134 ceph-mon[116146]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:17:58.104 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:58 smithi157 ceph-mon[116945]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:17:58.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:58 smithi120 ceph-mon[79813]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:17:58.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.281+0000 7fe1bd91f700 1 -- 172.21.15.157:0/892251462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1b80fde10 msgr2=0x7fe1b80fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:17:58.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.281+0000 7fe1bd91f700 1 --2- 172.21.15.157:0/892251462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1b80fde10 0x7fe1b80fe260 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe1a00049c0 tx=0x7fe1a0009e30 comp rx=0 tx=0).stop 2023-06-23T01:17:58.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.281+0000 7fe1bd91f700 1 -- 172.21.15.157:0/892251462 shutdown_connections 2023-06-23T01:17:58.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.281+0000 7fe1bd91f700 1 --2- 172.21.15.157:0/892251462 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe1b80fcc00 0x7fe1b80fd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:58.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.281+0000 7fe1bd91f700 1 --2- 172.21.15.157:0/892251462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1b80fe7a0 0x7fe1b8105060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:58.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.281+0000 7fe1bd91f700 1 --2- 172.21.15.157:0/892251462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1b80fde10 0x7fe1b80fe260 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:58.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.281+0000 7fe1bd91f700 1 -- 172.21.15.157:0/892251462 >> 172.21.15.157:0/892251462 conn(0x7fe1b80f8320 msgr2=0x7fe1b80fa790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:17:58.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.281+0000 7fe1bd91f700 1 -- 172.21.15.157:0/892251462 shutdown_connections 2023-06-23T01:17:58.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.281+0000 7fe1bd91f700 1 -- 172.21.15.157:0/892251462 wait complete. 2023-06-23T01:17:58.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.281+0000 7fe1bd91f700 1 Processor -- start 2023-06-23T01:17:58.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.281+0000 7fe1bd91f700 1 -- start start 2023-06-23T01:17:58.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.281+0000 7fe1bd91f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1b80fcc00 0x7fe1b810d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:17:58.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.281+0000 7fe1bd91f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1b80fde10 0x7fe1b810dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:17:58.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.282+0000 7fe1bd91f700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe1b80fe7a0 0x7fe1b8114e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:17:58.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.282+0000 7fe1bd91f700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b8115430 con 0x7fe1b80fe7a0 2023-06-23T01:17:58.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.282+0000 7fe1bd91f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b81155a0 con 0x7fe1b80fcc00 2023-06-23T01:17:58.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.282+0000 7fe1bd91f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b8115760 con 0x7fe1b80fde10 2023-06-23T01:17:58.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.282+0000 7fe1b67fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1b80fde10 0x7fe1b810dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:17:58.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.282+0000 7fe1b67fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1b80fde10 0x7fe1b810dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39448/0 (socket says 172.21.15.157:39448) 2023-06-23T01:17:58.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.282+0000 7fe1b67fc700 1 -- 172.21.15.157:0/1862826596 learned_addr learned my addr 172.21.15.157:0/1862826596 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-06-23T01:17:58.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.282+0000 7fe1b6ffd700 1 --2- 172.21.15.157:0/1862826596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1b80fcc00 0x7fe1b810d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:17:58.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.282+0000 7fe1b67fc700 1 -- 172.21.15.157:0/1862826596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1b80fcc00 msgr2=0x7fe1b810d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:17:58.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.282+0000 7fe1b67fc700 1 --2- 172.21.15.157:0/1862826596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1b80fcc00 0x7fe1b810d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:58.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.282+0000 7fe1b77fe700 1 --2- 172.21.15.157:0/1862826596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe1b80fe7a0 0x7fe1b8114e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:17:58.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.282+0000 7fe1b67fc700 1 -- 172.21.15.157:0/1862826596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe1b80fe7a0 msgr2=0x7fe1b8114e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:17:58.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.282+0000 7fe1b67fc700 1 --2- 172.21.15.157:0/1862826596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe1b80fe7a0 0x7fe1b8114e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:58.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.282+0000 7fe1b67fc700 1 -- 172.21.15.157:0/1862826596 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a0005040 con 0x7fe1b80fde10 2023-06-23T01:17:58.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.282+0000 7fe1b77fe700 1 --2- 172.21.15.157:0/1862826596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe1b80fe7a0 0x7fe1b8114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T01:17:58.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.282+0000 7fe1b67fc700 1 --2- 172.21.15.157:0/1862826596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1b80fde10 0x7fe1b810dd90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0007f30 tx=0x7fe1a0009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:17:58.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.282+0000 7fe1af7fe700 1 -- 172.21.15.157:0/1862826596 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a000e040 con 0x7fe1b80fde10 2023-06-23T01:17:58.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.282+0000 7fe1af7fe700 1 -- 172.21.15.157:0/1862826596 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1a0009770 con 0x7fe1b80fde10 2023-06-23T01:17:58.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.282+0000 7fe1bd91f700 1 -- 172.21.15.157:0/1862826596 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1b81159e0 con 0x7fe1b80fde10 2023-06-23T01:17:58.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.282+0000 7fe1af7fe700 1 -- 172.21.15.157:0/1862826596 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a0013ea0 con 0x7fe1b80fde10 2023-06-23T01:17:58.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.282+0000 7fe1bd91f700 1 -- 172.21.15.157:0/1862826596 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1b8115ec0 con 0x7fe1b80fde10 2023-06-23T01:17:58.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.283+0000 7fe1bd91f700 1 -- 172.21.15.157:0/1862826596 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1b81040d0 con 0x7fe1b80fde10 2023-06-23T01:17:58.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.284+0000 7fe1af7fe700 1 -- 172.21.15.157:0/1862826596 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7fe1a0013590 con 0x7fe1b80fde10 2023-06-23T01:17:58.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.285+0000 7fe1af7fe700 1 --2- 172.21.15.157:0/1862826596 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fe1a409b680 0x7fe1a409db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:17:58.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.285+0000 7fe1af7fe700 1 -- 172.21.15.157:0/1862826596 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7fe1a00c6630 con 0x7fe1b80fde10 2023-06-23T01:17:58.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.285+0000 7fe1b6ffd700 1 --2- 172.21.15.157:0/1862826596 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fe1a409b680 0x7fe1a409db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:17:58.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.286+0000 7fe1b6ffd700 1 --2- 172.21.15.157:0/1862826596 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fe1a409b680 0x7fe1a409db30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80fdc70 tx=0x7fe1b0009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:17:58.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.288+0000 7fe1af7fe700 1 -- 172.21.15.157:0/1862826596 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fe1a0092330 con 0x7fe1b80fde10 2023-06-23T01:17:58.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:58 smithi134 ceph-mon[116146]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:17:58.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.478+0000 7fe1bd91f700 1 -- 172.21.15.157:0/1862826596 --> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi120=a;smithi120=d;smithi120=g;smithi120=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l", "target": ["mon-mgr", ""]}) v1 -- 0x7fe1b802b280 con 0x7fe1a409b680 2023-06-23T01:17:58.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.487+0000 7fe1af7fe700 1 -- 172.21.15.157:0/1862826596 <== mgr.14152 v2:172.21.15.120:6800/1770912857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fe1b802b280 con 0x7fe1a409b680 2023-06-23T01:17:58.490 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled mds.all update... 2023-06-23T01:17:58.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.489+0000 7fe1bd91f700 1 -- 172.21.15.157:0/1862826596 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fe1a409b680 msgr2=0x7fe1a409db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:17:58.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.489+0000 7fe1bd91f700 1 --2- 172.21.15.157:0/1862826596 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fe1a409b680 0x7fe1a409db30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80fdc70 tx=0x7fe1b0009040 comp rx=0 tx=0).stop 2023-06-23T01:17:58.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.489+0000 7fe1bd91f700 1 -- 172.21.15.157:0/1862826596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1b80fde10 msgr2=0x7fe1b810dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:17:58.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.489+0000 7fe1bd91f700 1 --2- 172.21.15.157:0/1862826596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1b80fde10 0x7fe1b810dd90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0007f30 tx=0x7fe1a0009e30 comp rx=0 tx=0).stop 2023-06-23T01:17:58.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.489+0000 7fe1bd91f700 1 -- 172.21.15.157:0/1862826596 shutdown_connections 2023-06-23T01:17:58.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.489+0000 7fe1bd91f700 1 --2- 172.21.15.157:0/1862826596 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fe1a409b680 0x7fe1a409db30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:58.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.489+0000 7fe1bd91f700 1 --2- 172.21.15.157:0/1862826596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe1b80fe7a0 0x7fe1b8114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:58.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.489+0000 7fe1bd91f700 1 --2- 172.21.15.157:0/1862826596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1b80fcc00 0x7fe1b810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:58.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.489+0000 7fe1bd91f700 1 --2- 172.21.15.157:0/1862826596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1b80fde10 0x7fe1b810dd90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:58.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.489+0000 7fe1bd91f700 1 -- 172.21.15.157:0/1862826596 >> 172.21.15.157:0/1862826596 conn(0x7fe1b80f8320 msgr2=0x7fe1b8101250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:17:58.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.489+0000 7fe1bd91f700 1 -- 172.21.15.157:0/1862826596 shutdown_connections 2023-06-23T01:17:58.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-06-23T01:17:58.489+0000 7fe1bd91f700 1 -- 172.21.15.157:0/1862826596 wait complete. 2023-06-23T01:17:59.310 DEBUG:teuthology.orchestra.run.smithi120:mds.a> sudo journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mds.a.service 2023-06-23T01:17:59.315 DEBUG:teuthology.orchestra.run.smithi120:mds.d> sudo journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mds.d.service 2023-06-23T01:17:59.361 DEBUG:teuthology.orchestra.run.smithi120:mds.g> sudo journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mds.g.service 2023-06-23T01:17:59.383 INFO:journalctl@ceph.mds.a.smithi120.stdout:-- Logs begin at Fri 2023-06-23 00:53:53 UTC. -- 2023-06-23T01:17:59.387 DEBUG:teuthology.orchestra.run.smithi120:mds.j> sudo journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mds.j.service 2023-06-23T01:17:59.426 INFO:journalctl@ceph.mds.d.smithi120.stdout:-- Logs begin at Fri 2023-06-23 00:53:53 UTC. -- 2023-06-23T01:17:59.429 DEBUG:teuthology.orchestra.run.smithi134:mds.b> sudo journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mds.b.service 2023-06-23T01:17:59.431 DEBUG:teuthology.orchestra.run.smithi134:mds.e> sudo journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mds.e.service 2023-06-23T01:17:59.449 INFO:journalctl@ceph.mds.g.smithi120.stdout:-- Logs begin at Fri 2023-06-23 00:53:53 UTC. -- 2023-06-23T01:17:59.461 INFO:journalctl@ceph.mds.b.smithi134.stdout:-- Logs begin at Fri 2023-06-23 00:49:19 UTC. -- 2023-06-23T01:17:59.463 DEBUG:teuthology.orchestra.run.smithi134:mds.h> sudo journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mds.h.service 2023-06-23T01:17:59.486 INFO:journalctl@ceph.mds.e.smithi134.stdout:-- Logs begin at Fri 2023-06-23 00:49:19 UTC. -- 2023-06-23T01:17:59.486 DEBUG:teuthology.orchestra.run.smithi134:mds.k> sudo journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mds.k.service 2023-06-23T01:17:59.488 INFO:journalctl@ceph.mds.j.smithi120.stdout:-- Logs begin at Fri 2023-06-23 00:53:53 UTC. -- 2023-06-23T01:17:59.510 INFO:journalctl@ceph.mds.h.smithi134.stdout:-- Logs begin at Fri 2023-06-23 00:49:19 UTC. -- 2023-06-23T01:17:59.511 DEBUG:teuthology.orchestra.run.smithi157:mds.c> sudo journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mds.c.service 2023-06-23T01:17:59.513 DEBUG:teuthology.orchestra.run.smithi157:mds.f> sudo journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mds.f.service 2023-06-23T01:17:59.533 INFO:journalctl@ceph.mds.k.smithi134.stdout:-- Logs begin at Fri 2023-06-23 00:49:19 UTC. -- 2023-06-23T01:17:59.534 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:59 smithi157 ceph-mon[116945]: from='client.24448 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi120=a;smithi120=d;smithi120=g;smithi120=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:17:59.534 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:59 smithi157 ceph-mon[116945]: Saving service mds.all spec with placement smithi120=a;smithi120=d;smithi120=g;smithi120=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l;count:12 2023-06-23T01:17:59.534 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:59 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:59.534 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:59 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:17:59.534 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:59 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:17:59.535 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:59 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:17:59.535 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:59 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:59.535 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:59 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T01:17:59.535 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:59 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T01:17:59.535 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:17:59 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:17:59.536 DEBUG:teuthology.orchestra.run.smithi157:mds.i> sudo journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mds.i.service 2023-06-23T01:17:59.541 INFO:journalctl@ceph.mds.c.smithi157.stdout:-- Logs begin at Fri 2023-06-23 00:49:55 UTC. -- 2023-06-23T01:17:59.542 DEBUG:teuthology.orchestra.run.smithi157:mds.l> sudo journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mds.l.service 2023-06-23T01:17:59.560 INFO:journalctl@ceph.mds.f.smithi157.stdout:-- Logs begin at Fri 2023-06-23 00:49:55 UTC. -- 2023-06-23T01:17:59.561 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-06-23T01:17:59.562 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T01:17:59.566 INFO:journalctl@ceph.mds.i.smithi157.stdout:-- Logs begin at Fri 2023-06-23 00:49:55 UTC. -- 2023-06-23T01:17:59.587 INFO:journalctl@ceph.mds.l.smithi157.stdout:-- Logs begin at Fri 2023-06-23 00:49:55 UTC. -- 2023-06-23T01:17:59.760 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:59 smithi120 ceph-mon[79813]: from='client.24448 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi120=a;smithi120=d;smithi120=g;smithi120=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:17:59.760 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:59 smithi120 ceph-mon[79813]: Saving service mds.all spec with placement smithi120=a;smithi120=d;smithi120=g;smithi120=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l;count:12 2023-06-23T01:17:59.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:59 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:59.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:59 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:17:59.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:59 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:17:59.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:59 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:17:59.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:59 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:59.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:59 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T01:17:59.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:59 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T01:17:59.762 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:17:59 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:17:59.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:59 smithi134 ceph-mon[116146]: from='client.24448 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi120=a;smithi120=d;smithi120=g;smithi120=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T01:17:59.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:59 smithi134 ceph-mon[116146]: Saving service mds.all spec with placement smithi120=a;smithi120=d;smithi120=g;smithi120=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l;count:12 2023-06-23T01:17:59.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:59 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:59.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:59 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:17:59.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:59 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:17:59.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:59 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:17:59.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:59 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:17:59.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:59 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T01:17:59.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:59 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T01:17:59.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:17:59 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:17:59.971 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.968+0000 7fbe95b9d700 1 -- 172.21.15.120:0/481993438 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe900fe720 msgr2=0x7fbe90104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:17:59.972 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.968+0000 7fbe95b9d700 1 --2- 172.21.15.120:0/481993438 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe900fe720 0x7fbe90104fc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbe88009d60 tx=0x7fbe88004d10 comp rx=0 tx=0).stop 2023-06-23T01:17:59.972 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.968+0000 7fbe95b9d700 1 -- 172.21.15.120:0/481993438 shutdown_connections 2023-06-23T01:17:59.972 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.968+0000 7fbe95b9d700 1 --2- 172.21.15.120:0/481993438 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe900fe720 0x7fbe90104fc0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:59.972 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.968+0000 7fbe95b9d700 1 --2- 172.21.15.120:0/481993438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe900fcac0 0x7fbe900fced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:59.972 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.968+0000 7fbe95b9d700 1 --2- 172.21.15.120:0/481993438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe900fdd70 0x7fbe900fe1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:59.972 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.968+0000 7fbe95b9d700 1 -- 172.21.15.120:0/481993438 >> 172.21.15.120:0/481993438 conn(0x7fbe900f8240 msgr2=0x7fbe900fa650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:17:59.973 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.968+0000 7fbe95b9d700 1 -- 172.21.15.120:0/481993438 shutdown_connections 2023-06-23T01:17:59.973 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.968+0000 7fbe95b9d700 1 -- 172.21.15.120:0/481993438 wait complete. 2023-06-23T01:17:59.975 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.969+0000 7fbe95b9d700 1 Processor -- start 2023-06-23T01:17:59.975 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.969+0000 7fbe95b9d700 1 -- start start 2023-06-23T01:17:59.975 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.969+0000 7fbe95b9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe900fcac0 0x7fbe90071720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:17:59.975 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.970+0000 7fbe95b9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe900fdd70 0x7fbe9006fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:17:59.975 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.970+0000 7fbe95b9d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe900fe720 0x7fbe900702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:17:59.975 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.970+0000 7fbe95b9d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe90070a30 con 0x7fbe900fe720 2023-06-23T01:17:59.976 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.970+0000 7fbe95b9d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe90070ba0 con 0x7fbe900fdd70 2023-06-23T01:17:59.976 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.970+0000 7fbe95b9d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe90117630 con 0x7fbe900fcac0 2023-06-23T01:17:59.976 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.970+0000 7fbe9539c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe900fe720 0x7fbe900702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:17:59.976 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.970+0000 7fbe94b9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe900fcac0 0x7fbe90071720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:17:59.976 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.970+0000 7fbe94b9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe900fcac0 0x7fbe90071720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.120:35374/0 (socket says 172.21.15.120:35374) 2023-06-23T01:17:59.976 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.970+0000 7fbe9539c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe900fe720 0x7fbe900702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50162/0 (socket says 172.21.15.120:50162) 2023-06-23T01:17:59.977 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.970+0000 7fbe94b9b700 1 -- 172.21.15.120:0/987474304 learned_addr learned my addr 172.21.15.120:0/987474304 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:17:59.977 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.970+0000 7fbe8ffff700 1 --2- 172.21.15.120:0/987474304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe900fdd70 0x7fbe9006fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:17:59.977 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.970+0000 7fbe9539c700 1 -- 172.21.15.120:0/987474304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe900fcac0 msgr2=0x7fbe90071720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:17:59.977 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.970+0000 7fbe9539c700 1 --2- 172.21.15.120:0/987474304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe900fcac0 0x7fbe90071720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:59.978 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.970+0000 7fbe9539c700 1 -- 172.21.15.120:0/987474304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe900fdd70 msgr2=0x7fbe9006fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:17:59.978 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.970+0000 7fbe9539c700 1 --2- 172.21.15.120:0/987474304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe900fdd70 0x7fbe9006fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:17:59.978 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.970+0000 7fbe9539c700 1 -- 172.21.15.120:0/987474304 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe88009940 con 0x7fbe900fe720 2023-06-23T01:17:59.978 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.970+0000 7fbe9539c700 1 --2- 172.21.15.120:0/987474304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe900fe720 0x7fbe900702c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbe88009140 tx=0x7fbe88004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:17:59.978 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.971+0000 7fbe8d7fa700 1 -- 172.21.15.120:0/987474304 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8800e070 con 0x7fbe900fe720 2023-06-23T01:17:59.978 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.971+0000 7fbe8d7fa700 1 -- 172.21.15.120:0/987474304 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe880097c0 con 0x7fbe900fe720 2023-06-23T01:17:59.978 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.971+0000 7fbe95b9d700 1 -- 172.21.15.120:0/987474304 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe90117880 con 0x7fbe900fe720 2023-06-23T01:17:59.978 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.971+0000 7fbe8d7fa700 1 -- 172.21.15.120:0/987474304 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8801c7f0 con 0x7fbe900fe720 2023-06-23T01:17:59.979 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.971+0000 7fbe95b9d700 1 -- 172.21.15.120:0/987474304 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe90117d60 con 0x7fbe900fe720 2023-06-23T01:17:59.979 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.972+0000 7fbe95b9d700 1 -- 172.21.15.120:0/987474304 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe90108320 con 0x7fbe900fe720 2023-06-23T01:17:59.979 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.973+0000 7fbe8d7fa700 1 -- 172.21.15.120:0/987474304 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7fbe88012830 con 0x7fbe900fe720 2023-06-23T01:17:59.980 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.974+0000 7fbe8d7fa700 1 --2- 172.21.15.120:0/987474304 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fbe8009b680 0x7fbe8009db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:17:59.980 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.975+0000 7fbe8d7fa700 1 -- 172.21.15.120:0/987474304 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7fbe880c6560 con 0x7fbe900fe720 2023-06-23T01:17:59.980 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.975+0000 7fbe94b9b700 1 --2- 172.21.15.120:0/987474304 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fbe8009b680 0x7fbe8009db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:17:59.980 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.975+0000 7fbe94b9b700 1 --2- 172.21.15.120:0/987474304 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fbe8009b680 0x7fbe8009db30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbe840041d0 tx=0x7fbe8400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:17:59.981 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:17:59.979+0000 7fbe8d7fa700 1 -- 172.21.15.120:0/987474304 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fbe880921c0 con 0x7fbe900fe720 2023-06-23T01:18:00.177 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.174+0000 7fbe95b9d700 1 -- 172.21.15.120:0/987474304 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe90117f80 con 0x7fbe900fe720 2023-06-23T01:18:00.179 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.175+0000 7fbe8d7fa700 1 -- 172.21.15.120:0/987474304 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18241 (secure 0 0 0) 0x7fbe880957e0 con 0x7fbe900fe720 2023-06-23T01:18:00.180 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:18:00.180 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":81,"fsid":"9efbfa8e-1162-11ee-9b2b-001a4aab830c","created":"2023-06-23T01:10:46.840474+0000","modified":"2023-06-23T01:17:47.828105+0000","last_up_change":"2023-06-23T01:17:46.670887+0000","last_in_change":"2023-06-23T01:17:27.552330+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T01:14:03.589464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"3729642b-e74f-4253-bde8-535d8bfe8c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3479894866},{"type":"v1","addr":"172.21.15.120:6803","nonce":3479894866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3479894866},{"type":"v1","addr":"172.21.15.120:6805","nonce":3479894866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3479894866},{"type":"v1","addr":"172.21.15.120:6809","nonce":3479894866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3479894866},{"type":"v1","addr":"172.21.15.120:6807","nonce":3479894866}]},"public_addr":"172.21.15.120:6803/3479894866","cluster_addr":"172.21.15.120:6805/3479894866","heartbeat_back_addr":"172.21.15.120:6809/3479894866","heartbeat_front_addr":"172.21.15.120:6807/3479894866","state":["exists","up"]},{"osd":1,"uuid":"58240258-afd5-4991-9d5b-a80d365e4fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3046662876},{"type":"v1","addr":"172.21.15.134:6801","nonce":3046662876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3046662876},{"type":"v1","addr":"172.21.15.134:6803","nonce":3046662876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3046662876},{"type":"v1","addr":"172.21.15.134:6807","nonce":3046662876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3046662876},{"type":"v1","addr":"172.21.15.134:6805","nonce":3046662876}]},"public_addr":"172.21.15.134:6801/3046662876","cluster_addr":"172.21.15.134:6803/3046662876","heartbeat_back_addr":"172.21.15.134:6807/3046662876","heartbeat_front_addr":"172.21.15.134:6805/3046662876","state":["exists","up"]},{"osd":2,"uuid":"05988a7a-7cf2-4fbb-9d4f-22604cd2f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3996265223},{"type":"v1","addr":"172.21.15.157:6801","nonce":3996265223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3996265223},{"type":"v1","addr":"172.21.15.157:6803","nonce":3996265223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3996265223},{"type":"v1","addr":"172.21.15.157:6807","nonce":3996265223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3996265223},{"type":"v1","addr":"172.21.15.157:6805","nonce":3996265223}]},"public_addr":"172.21.15.157:6801/3996265223","cluster_addr":"172.21.15.157:6803/3996265223","heartbeat_back_addr":"172.21.15.157:6807/3996265223","heartbeat_front_addr":"172.21.15.157:6805/3996265223","state":["exists","up"]},{"osd":3,"uuid":"7d9e9739-f043-43d6-8e46-7711dcf43e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":1716638437},{"type":"v1","addr":"172.21.15.120:6811","nonce":1716638437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":1716638437},{"type":"v1","addr":"172.21.15.120:6813","nonce":1716638437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1716638437},{"type":"v1","addr":"172.21.15.120:6817","nonce":1716638437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":1716638437},{"type":"v1","addr":"172.21.15.120:6815","nonce":1716638437}]},"public_addr":"172.21.15.120:6811/1716638437","cluster_addr":"172.21.15.120:6813/1716638437","heartbeat_back_addr":"172.21.15.120:6817/1716638437","heartbeat_front_addr":"172.21.15.120:6815/1716638437","state":["exists","up"]},{"osd":4,"uuid":"3a70f86d-f903-4618-be66-9aa1eac44ccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4172238262},{"type":"v1","addr":"172.21.15.134:6809","nonce":4172238262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4172238262},{"type":"v1","addr":"172.21.15.134:6811","nonce":4172238262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4172238262},{"type":"v1","addr":"172.21.15.134:6815","nonce":4172238262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4172238262},{"type":"v1","addr":"172.21.15.134:6813","nonce":4172238262}]},"public_addr":"172.21.15.134:6809/4172238262","cluster_addr":"172.21.15.134:6811/4172238262","heartbeat_back_addr":"172.21.15.134:6815/4172238262","heartbeat_front_addr":"172.21.15.134:6813/4172238262","state":["exists","up"]},{"osd":5,"uuid":"b2316488-75c3-4847-b4c4-998730f27c88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3374822045},{"type":"v1","addr":"172.21.15.157:6809","nonce":3374822045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3374822045},{"type":"v1","addr":"172.21.15.157:6811","nonce":3374822045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3374822045},{"type":"v1","addr":"172.21.15.157:6815","nonce":3374822045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3374822045},{"type":"v1","addr":"172.21.15.157:6813","nonce":3374822045}]},"public_addr":"172.21.15.157:6809/3374822045","cluster_addr":"172.21.15.157:6811/3374822045","heartbeat_back_addr":"172.21.15.157:6815/3374822045","heartbeat_front_addr":"172.21.15.157:6813/3374822045","state":["exists","up"]},{"osd":6,"uuid":"338c2c4a-d982-4031-aaaf-be76c9dd74a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2097345225},{"type":"v1","addr":"172.21.15.120:6819","nonce":2097345225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2097345225},{"type":"v1","addr":"172.21.15.120:6821","nonce":2097345225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2097345225},{"type":"v1","addr":"172.21.15.120:6825","nonce":2097345225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2097345225},{"type":"v1","addr":"172.21.15.120:6823","nonce":2097345225}]},"public_addr":"172.21.15.120:6819/2097345225","cluster_addr":"172.21.15.120:6821/2097345225","heartbeat_back_addr":"172.21.15.120:6825/2097345225","heartbeat_front_addr":"172.21.15.120:6823/2097345225","state":["exists","up"]},{"osd":7,"uuid":"bff92799-ae9c-4409-be65-7ffc27a1c3f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1345745298},{"type":"v1","addr":"172.21.15.134:6817","nonce":1345745298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1345745298},{"type":"v1","addr":"172.21.15.134:6819","nonce":1345745298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1345745298},{"type":"v1","addr":"172.21.15.134:6823","nonce":1345745298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1345745298},{"type":"v1","addr":"172.21.15.134:6821","nonce":1345745298}]},"public_addr":"172.21.15.134:6817/1345745298","cluster_addr":"172.21.15.134:6819/1345745298","heartbeat_back_addr":"172.21.15.134:6823/1345745298","heartbeat_front_addr":"172.21.15.134:6821/1345745298","state":["exists","up"]},{"osd":8,"uuid":"e4aac47e-d7f9-4feb-ada7-d8103fdca246","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1124549599},{"type":"v1","addr":"172.21.15.157:6817","nonce":1124549599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1124549599},{"type":"v1","addr":"172.21.15.157:6819","nonce":1124549599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1124549599},{"type":"v1","addr":"172.21.15.157:6823","nonce":1124549599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1124549599},{"type":"v1","addr":"172.21.15.157:6821","nonce":1124549599}]},"public_addr":"172.21.15.157:6817/1124549599","cluster_addr":"172.21.15.157:6819/1124549599","heartbeat_back_addr":"172.21.15.157:6823/1124549599","heartbeat_front_addr":"172.21.15.157:6821/1124549599","state":["exists","up"]},{"osd":9,"uuid":"48b988ba-e866-47d1-aca9-09f6ad815bd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1402414981},{"type":"v1","addr":"172.21.15.120:6827","nonce":1402414981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1402414981},{"type":"v1","addr":"172.21.15.120:6829","nonce":1402414981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1402414981},{"type":"v1","addr":"172.21.15.120:6833","nonce":1402414981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1402414981},{"type":"v1","addr":"172.21.15.120:6831","nonce":1402414981}]},"public_addr":"172.21.15.120:6827/1402414981","cluster_addr":"172.21.15.120:6829/1402414981","heartbeat_back_addr":"172.21.15.120:6833/1402414981","heartbeat_front_addr":"172.21.15.120:6831/1402414981","state":["exists","up"]},{"osd":10,"uuid":"777f3add-3779-4221-a86f-65cca2e5e8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":4041441144},{"type":"v1","addr":"172.21.15.134:6825","nonce":4041441144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":4041441144},{"type":"v1","addr":"172.21.15.134:6827","nonce":4041441144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":4041441144},{"type":"v1","addr":"172.21.15.134:6831","nonce":4041441144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":4041441144},{"type":"v1","addr":"172.21.15.134:6829","nonce":4041441144}]},"public_addr":"172.21.15.134:6825/4041441144","cluster_addr":"172.21.15.134:6827/4041441144","heartbeat_back_addr":"172.21.15.134:6831/4041441144","heartbeat_front_addr":"172.21.15.134:6829/4041441144","state":["exists","up"]},{"osd":11,"uuid":"7166e11b-03c0-45bf-abbf-a74c4a13a39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1537477260},{"type":"v1","addr":"172.21.15.157:6825","nonce":1537477260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1537477260},{"type":"v1","addr":"172.21.15.157:6827","nonce":1537477260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1537477260},{"type":"v1","addr":"172.21.15.157:6831","nonce":1537477260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1537477260},{"type":"v1","addr":"172.21.15.157:6829","nonce":1537477260}]},"public_addr":"172.21.15.157:6825/1537477260","cluster_addr":"172.21.15.157:6827/1537477260","heartbeat_back_addr":"172.21.15.157:6831/1537477260","heartbeat_front_addr":"172.21.15.157:6829/1537477260","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:13:13.580631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:13:37.472924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:14:01.591555+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:14:28.472130+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:14:52.356433+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:15:15.946523+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:15:42.036105+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:16:06.154934+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:16:30.043788+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:16:55.975403+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:17:20.612778+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:17:44.896301+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.120:0/2865552846":"2023-06-24T01:11:38.967204+0000","172.21.15.120:0/2163781512":"2023-06-24T01:11:38.967204+0000","172.21.15.120:0/1943345772":"2023-06-24T01:11:38.967204+0000","172.21.15.120:6801/1836713308":"2023-06-24T01:11:38.967204+0000","172.21.15.120:0/2358086231":"2023-06-24T01:11:14.574788+0000","172.21.15.120:6801/506843621":"2023-06-24T01:11:14.574788+0000","172.21.15.120:6800/506843621":"2023-06-24T01:11:14.574788+0000","172.21.15.120:0/4077038498":"2023-06-24T01:11:14.574788+0000","172.21.15.120:6800/1836713308":"2023-06-24T01:11:38.967204+0000","172.21.15.120:0/2484415475":"2023-06-24T01:11:14.574788+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T01:18:00.182 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.177+0000 7fbe95b9d700 1 -- 172.21.15.120:0/987474304 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fbe8009b680 msgr2=0x7fbe8009db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:00.183 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.177+0000 7fbe95b9d700 1 --2- 172.21.15.120:0/987474304 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fbe8009b680 0x7fbe8009db30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbe840041d0 tx=0x7fbe8400b040 comp rx=0 tx=0).stop 2023-06-23T01:18:00.183 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.177+0000 7fbe95b9d700 1 -- 172.21.15.120:0/987474304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe900fe720 msgr2=0x7fbe900702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:00.183 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.177+0000 7fbe95b9d700 1 --2- 172.21.15.120:0/987474304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe900fe720 0x7fbe900702c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbe88009140 tx=0x7fbe88004d10 comp rx=0 tx=0).stop 2023-06-23T01:18:00.183 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.178+0000 7fbe95b9d700 1 -- 172.21.15.120:0/987474304 shutdown_connections 2023-06-23T01:18:00.184 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.178+0000 7fbe95b9d700 1 --2- 172.21.15.120:0/987474304 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fbe8009b680 0x7fbe8009db30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:00.184 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.178+0000 7fbe95b9d700 1 --2- 172.21.15.120:0/987474304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe900fe720 0x7fbe900702c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:00.184 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.178+0000 7fbe95b9d700 1 --2- 172.21.15.120:0/987474304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe900fcac0 0x7fbe90071720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:00.184 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.178+0000 7fbe95b9d700 1 --2- 172.21.15.120:0/987474304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe900fdd70 0x7fbe9006fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:00.184 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.178+0000 7fbe95b9d700 1 -- 172.21.15.120:0/987474304 >> 172.21.15.120:0/987474304 conn(0x7fbe900f8240 msgr2=0x7fbe900fa1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:00.185 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.178+0000 7fbe95b9d700 1 -- 172.21.15.120:0/987474304 shutdown_connections 2023-06-23T01:18:00.185 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.178+0000 7fbe95b9d700 1 -- 172.21.15.120:0/987474304 wait complete. 2023-06-23T01:18:00.200 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-23T01:14:03.589464+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-06-23T01:18:00.201 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T01:18:00.386 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.383+0000 7f390a79d700 1 -- 172.21.15.120:0/2847518111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39040fe680 msgr2=0x7f3904104f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:00.386 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.383+0000 7f390a79d700 1 --2- 172.21.15.120:0/2847518111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39040fe680 0x7f3904104f40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0049c0 tx=0x7f38fc009e30 comp rx=0 tx=0).stop 2023-06-23T01:18:00.386 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.383+0000 7f390a79d700 1 -- 172.21.15.120:0/2847518111 shutdown_connections 2023-06-23T01:18:00.386 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.383+0000 7f390a79d700 1 --2- 172.21.15.120:0/2847518111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39040fe680 0x7f3904104f40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:00.387 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.383+0000 7f390a79d700 1 --2- 172.21.15.120:0/2847518111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39040fcae0 0x7f39040fcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:00.387 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.383+0000 7f390a79d700 1 --2- 172.21.15.120:0/2847518111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39040fdcf0 0x7f39040fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:00.387 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.383+0000 7f390a79d700 1 -- 172.21.15.120:0/2847518111 >> 172.21.15.120:0/2847518111 conn(0x7f39040f8200 msgr2=0x7f39040fa670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:00.387 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.383+0000 7f390a79d700 1 -- 172.21.15.120:0/2847518111 shutdown_connections 2023-06-23T01:18:00.387 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.383+0000 7f390a79d700 1 -- 172.21.15.120:0/2847518111 wait complete. 2023-06-23T01:18:00.389 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.384+0000 7f390a79d700 1 Processor -- start 2023-06-23T01:18:00.389 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.384+0000 7f390a79d700 1 -- start start 2023-06-23T01:18:00.389 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.384+0000 7f390a79d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39040fcae0 0x7f3904071720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:00.389 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.384+0000 7f390a79d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39040fdcf0 0x7f390406fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:00.389 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.384+0000 7f390a79d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39040fe680 0x7f39040702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:00.390 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.384+0000 7f390a79d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3904070a30 con 0x7f39040fdcf0 2023-06-23T01:18:00.390 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.384+0000 7f390a79d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3904070ba0 con 0x7f39040fe680 2023-06-23T01:18:00.390 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.384+0000 7f390a79d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3904117580 con 0x7f39040fcae0 2023-06-23T01:18:00.390 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.384+0000 7f3908f9a700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39040fdcf0 0x7f390406fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:00.390 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.384+0000 7f3908f9a700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39040fdcf0 0x7f390406fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50178/0 (socket says 172.21.15.120:50178) 2023-06-23T01:18:00.390 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.384+0000 7f3909f9c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39040fe680 0x7f39040702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:00.390 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.384+0000 7f3908f9a700 1 -- 172.21.15.120:0/2972691954 learned_addr learned my addr 172.21.15.120:0/2972691954 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:18:00.390 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.384+0000 7f390979b700 1 --2- 172.21.15.120:0/2972691954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39040fcae0 0x7f3904071720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:00.391 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.385+0000 7f3908f9a700 1 -- 172.21.15.120:0/2972691954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39040fcae0 msgr2=0x7f3904071720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:00.391 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.385+0000 7f3908f9a700 1 --2- 172.21.15.120:0/2972691954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39040fcae0 0x7f3904071720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:00.391 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.385+0000 7f3908f9a700 1 -- 172.21.15.120:0/2972691954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39040fe680 msgr2=0x7f39040702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:00.391 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.385+0000 7f3908f9a700 1 --2- 172.21.15.120:0/2972691954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39040fe680 0x7f39040702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:00.391 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.385+0000 7f3908f9a700 1 -- 172.21.15.120:0/2972691954 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38fc005040 con 0x7f39040fdcf0 2023-06-23T01:18:00.391 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.385+0000 7f3908f9a700 1 --2- 172.21.15.120:0/2972691954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39040fdcf0 0x7f390406fd80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f38f4002b00 tx=0x7f38f40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:00.391 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.385+0000 7f38f9ffb700 1 -- 172.21.15.120:0/2972691954 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f4007cc0 con 0x7f39040fdcf0 2023-06-23T01:18:00.392 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.385+0000 7f38f9ffb700 1 -- 172.21.15.120:0/2972691954 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38f400f040 con 0x7f39040fdcf0 2023-06-23T01:18:00.392 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.385+0000 7f390a79d700 1 -- 172.21.15.120:0/2972691954 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3904117830 con 0x7f39040fdcf0 2023-06-23T01:18:00.392 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.385+0000 7f38f9ffb700 1 -- 172.21.15.120:0/2972691954 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f400d990 con 0x7f39040fdcf0 2023-06-23T01:18:00.392 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.385+0000 7f390a79d700 1 -- 172.21.15.120:0/2972691954 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3904117d20 con 0x7f39040fdcf0 2023-06-23T01:18:00.392 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.386+0000 7f390a79d700 1 -- 172.21.15.120:0/2972691954 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3904103da0 con 0x7f39040fdcf0 2023-06-23T01:18:00.393 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.388+0000 7f38f9ffb700 1 -- 172.21.15.120:0/2972691954 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7f38f401c400 con 0x7f39040fdcf0 2023-06-23T01:18:00.393 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.389+0000 7f38f9ffb700 1 --2- 172.21.15.120:0/2972691954 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f38f009b680 0x7f38f009db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:00.393 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.389+0000 7f38f9ffb700 1 -- 172.21.15.120:0/2972691954 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f38f40c4780 con 0x7f39040fdcf0 2023-06-23T01:18:00.393 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.390+0000 7f390979b700 1 --2- 172.21.15.120:0/2972691954 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f38f009b680 0x7f38f009db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:00.394 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.390+0000 7f390979b700 1 --2- 172.21.15.120:0/2972691954 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f38f009b680 0x7f38f009db30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f39040fdb50 tx=0x7f390000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:00.396 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.393+0000 7f38f9ffb700 1 -- 172.21.15.120:0/2972691954 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f38f40903e0 con 0x7f39040fdcf0 2023-06-23T01:18:00.591 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.587+0000 7f390a79d700 1 -- 172.21.15.120:0/2972691954 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f390402b250 con 0x7f39040fdcf0 2023-06-23T01:18:00.591 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.588+0000 7f38f9ffb700 1 -- 172.21.15.120:0/2972691954 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f38f4093a00 con 0x7f39040fdcf0 2023-06-23T01:18:00.591 INFO:teuthology.orchestra.run.smithi120.stdout:pg_num: 1 2023-06-23T01:18:00.593 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.590+0000 7f390a79d700 1 -- 172.21.15.120:0/2972691954 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f38f009b680 msgr2=0x7f38f009db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:00.593 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.590+0000 7f390a79d700 1 --2- 172.21.15.120:0/2972691954 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f38f009b680 0x7f38f009db30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f39040fdb50 tx=0x7f390000b040 comp rx=0 tx=0).stop 2023-06-23T01:18:00.593 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.590+0000 7f390a79d700 1 -- 172.21.15.120:0/2972691954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39040fdcf0 msgr2=0x7f390406fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:00.594 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.590+0000 7f390a79d700 1 --2- 172.21.15.120:0/2972691954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39040fdcf0 0x7f390406fd80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f38f4002b00 tx=0x7f38f40046b0 comp rx=0 tx=0).stop 2023-06-23T01:18:00.594 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.590+0000 7f390a79d700 1 -- 172.21.15.120:0/2972691954 shutdown_connections 2023-06-23T01:18:00.594 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.590+0000 7f390a79d700 1 --2- 172.21.15.120:0/2972691954 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f38f009b680 0x7f38f009db30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:00.594 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.590+0000 7f390a79d700 1 --2- 172.21.15.120:0/2972691954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39040fe680 0x7f39040702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:00.594 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.590+0000 7f390a79d700 1 --2- 172.21.15.120:0/2972691954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39040fcae0 0x7f3904071720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:00.594 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.590+0000 7f390a79d700 1 --2- 172.21.15.120:0/2972691954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39040fdcf0 0x7f390406fd80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:00.594 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.590+0000 7f390a79d700 1 -- 172.21.15.120:0/2972691954 >> 172.21.15.120:0/2972691954 conn(0x7f39040f8200 msgr2=0x7f3904100f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:00.595 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.590+0000 7f390a79d700 1 -- 172.21.15.120:0/2972691954 shutdown_connections 2023-06-23T01:18:00.595 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.590+0000 7f390a79d700 1 -- 172.21.15.120:0/2972691954 wait complete. 2023-06-23T01:18:00.612 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T01:18:00.760 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:00 smithi120 ceph-mon[79813]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:00.760 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:00 smithi120 ceph-mon[79813]: Deploying daemon mds.l on smithi157 2023-06-23T01:18:00.760 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:00 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/987474304' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T01:18:00.796 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.793+0000 7fe27339d700 1 -- 172.21.15.120:0/1262544012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe26c071890 msgr2=0x7fe26c06fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:00.796 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.793+0000 7fe27339d700 1 --2- 172.21.15.120:0/1262544012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe26c071890 0x7fe26c06fda0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe2680049c0 tx=0x7fe268009e30 comp rx=0 tx=0).stop 2023-06-23T01:18:00.797 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.793+0000 7fe27339d700 1 -- 172.21.15.120:0/1262544012 shutdown_connections 2023-06-23T01:18:00.797 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.793+0000 7fe27339d700 1 --2- 172.21.15.120:0/1262544012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe26c104a80 0x7fe26c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:00.797 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.793+0000 7fe27339d700 1 --2- 172.21.15.120:0/1262544012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe26c071890 0x7fe26c06fda0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:00.797 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.793+0000 7fe27339d700 1 --2- 172.21.15.120:0/1262544012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe26c0702e0 0x7fe26c070750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:00.797 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.793+0000 7fe27339d700 1 -- 172.21.15.120:0/1262544012 >> 172.21.15.120:0/1262544012 conn(0x7fe26c0f8240 msgr2=0x7fe26c0fa650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:00.797 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.793+0000 7fe27339d700 1 -- 172.21.15.120:0/1262544012 shutdown_connections 2023-06-23T01:18:00.798 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.793+0000 7fe27339d700 1 -- 172.21.15.120:0/1262544012 wait complete. 2023-06-23T01:18:00.799 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.794+0000 7fe27339d700 1 Processor -- start 2023-06-23T01:18:00.799 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.794+0000 7fe27339d700 1 -- start start 2023-06-23T01:18:00.800 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.794+0000 7fe27339d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe26c0702e0 0x7fe26c10f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:00.800 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.794+0000 7fe27339d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe26c071890 0x7fe26c10fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:00.800 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.794+0000 7fe27339d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe26c104a80 0x7fe26c116f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:00.800 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.794+0000 7fe27339d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26c117530 con 0x7fe26c104a80 2023-06-23T01:18:00.800 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.794+0000 7fe27339d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26c1176a0 con 0x7fe26c0702e0 2023-06-23T01:18:00.801 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.794+0000 7fe27339d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26c117860 con 0x7fe26c071890 2023-06-23T01:18:00.801 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.794+0000 7fe272b9c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe26c104a80 0x7fe26c116f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:00.801 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.794+0000 7fe271b9a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe26c071890 0x7fe26c10fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:00.801 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.794+0000 7fe27239b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe26c0702e0 0x7fe26c10f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:00.801 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.794+0000 7fe271b9a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe26c071890 0x7fe26c10fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.120:35400/0 (socket says 172.21.15.120:35400) 2023-06-23T01:18:00.801 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.794+0000 7fe272b9c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe26c104a80 0x7fe26c116f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50198/0 (socket says 172.21.15.120:50198) 2023-06-23T01:18:00.802 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.794+0000 7fe27239b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe26c0702e0 0x7fe26c10f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:43080/0 (socket says 172.21.15.120:43080) 2023-06-23T01:18:00.802 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.794+0000 7fe272b9c700 1 -- 172.21.15.120:0/3615144072 learned_addr learned my addr 172.21.15.120:0/3615144072 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:18:00.802 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.795+0000 7fe27239b700 1 -- 172.21.15.120:0/3615144072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe26c071890 msgr2=0x7fe26c10fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:00.802 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.795+0000 7fe27239b700 1 --2- 172.21.15.120:0/3615144072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe26c071890 0x7fe26c10fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:00.802 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.795+0000 7fe27239b700 1 -- 172.21.15.120:0/3615144072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe26c104a80 msgr2=0x7fe26c116f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:00.802 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.795+0000 7fe27239b700 1 --2- 172.21.15.120:0/3615144072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe26c104a80 0x7fe26c116f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:00.803 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.795+0000 7fe27239b700 1 -- 172.21.15.120:0/3615144072 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe268005040 con 0x7fe26c0702e0 2023-06-23T01:18:00.803 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.795+0000 7fe272b9c700 1 --2- 172.21.15.120:0/3615144072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe26c104a80 0x7fe26c116f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T01:18:00.803 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.795+0000 7fe27239b700 1 --2- 172.21.15.120:0/3615144072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe26c0702e0 0x7fe26c10f950 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe2680049c0 tx=0x7fe268009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:00.803 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.795+0000 7fe262ffd700 1 -- 172.21.15.120:0/3615144072 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26800e050 con 0x7fe26c0702e0 2023-06-23T01:18:00.803 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.795+0000 7fe27339d700 1 -- 172.21.15.120:0/3615144072 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe26c117ae0 con 0x7fe26c0702e0 2023-06-23T01:18:00.803 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.795+0000 7fe262ffd700 1 -- 172.21.15.120:0/3615144072 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2680097b0 con 0x7fe26c0702e0 2023-06-23T01:18:00.804 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.796+0000 7fe262ffd700 1 -- 172.21.15.120:0/3615144072 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26801be60 con 0x7fe26c0702e0 2023-06-23T01:18:00.804 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.796+0000 7fe27339d700 1 -- 172.21.15.120:0/3615144072 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe26c117f90 con 0x7fe26c0702e0 2023-06-23T01:18:00.804 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.796+0000 7fe27339d700 1 -- 172.21.15.120:0/3615144072 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe26c1031d0 con 0x7fe26c0702e0 2023-06-23T01:18:00.805 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.799+0000 7fe262ffd700 1 -- 172.21.15.120:0/3615144072 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7fe26801b3f0 con 0x7fe26c0702e0 2023-06-23T01:18:00.805 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.800+0000 7fe262ffd700 1 --2- 172.21.15.120:0/3615144072 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fe25809b680 0x7fe25809db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:00.805 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.800+0000 7fe262ffd700 1 -- 172.21.15.120:0/3615144072 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7fe2680c5bf0 con 0x7fe26c0702e0 2023-06-23T01:18:00.805 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.800+0000 7fe271b9a700 1 --2- 172.21.15.120:0/3615144072 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fe25809b680 0x7fe25809db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:00.806 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.800+0000 7fe271b9a700 1 --2- 172.21.15.120:0/3615144072 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fe25809b680 0x7fe25809db30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe25c009b80 tx=0x7fe25c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:00.806 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:00.802+0000 7fe262ffd700 1 -- 172.21.15.120:0/3615144072 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fe268091850 con 0x7fe26c0702e0 2023-06-23T01:18:00.822 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:00 smithi157 ceph-mon[116945]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:00.823 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:00 smithi157 ceph-mon[116945]: Deploying daemon mds.l on smithi157 2023-06-23T01:18:00.823 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:00 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/987474304' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T01:18:00.943 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:00 smithi134 ceph-mon[116146]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:00.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:00 smithi134 ceph-mon[116146]: Deploying daemon mds.l on smithi157 2023-06-23T01:18:00.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:00 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/987474304' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T01:18:01.043 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:01.040+0000 7fe27339d700 1 -- 172.21.15.120:0/3615144072 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe26c10a530 con 0x7fe26c0702e0 2023-06-23T01:18:01.044 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:01.041+0000 7fe262ffd700 1 -- 172.21.15.120:0/3615144072 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7fe268028090 con 0x7fe26c0702e0 2023-06-23T01:18:01.044 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:18:01.044 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-06-23T01:18:01.046 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:01.043+0000 7fe27339d700 1 -- 172.21.15.120:0/3615144072 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fe25809b680 msgr2=0x7fe25809db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:01.047 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:01.043+0000 7fe27339d700 1 --2- 172.21.15.120:0/3615144072 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fe25809b680 0x7fe25809db30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe25c009b80 tx=0x7fe25c00b040 comp rx=0 tx=0).stop 2023-06-23T01:18:01.047 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:01.043+0000 7fe27339d700 1 -- 172.21.15.120:0/3615144072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe26c0702e0 msgr2=0x7fe26c10f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:01.047 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:01.043+0000 7fe27339d700 1 --2- 172.21.15.120:0/3615144072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe26c0702e0 0x7fe26c10f950 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe2680049c0 tx=0x7fe268009e30 comp rx=0 tx=0).stop 2023-06-23T01:18:01.047 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:01.043+0000 7fe27339d700 1 -- 172.21.15.120:0/3615144072 shutdown_connections 2023-06-23T01:18:01.047 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:01.043+0000 7fe27339d700 1 --2- 172.21.15.120:0/3615144072 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fe25809b680 0x7fe25809db30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:01.047 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:01.043+0000 7fe27339d700 1 --2- 172.21.15.120:0/3615144072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe26c104a80 0x7fe26c116f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:01.047 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:01.043+0000 7fe27339d700 1 --2- 172.21.15.120:0/3615144072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe26c0702e0 0x7fe26c10f950 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:01.047 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:01.043+0000 7fe27339d700 1 --2- 172.21.15.120:0/3615144072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe26c071890 0x7fe26c10fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:01.048 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:01.043+0000 7fe27339d700 1 -- 172.21.15.120:0/3615144072 >> 172.21.15.120:0/3615144072 conn(0x7fe26c0f8240 msgr2=0x7fe26c0fa620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:01.048 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:01.043+0000 7fe27339d700 1 -- 172.21.15.120:0/3615144072 shutdown_connections 2023-06-23T01:18:01.048 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:01.044+0000 7fe27339d700 1 -- 172.21.15.120:0/3615144072 wait complete. 2023-06-23T01:18:01.048 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 1 2023-06-23T01:18:01.126 INFO:journalctl@ceph.mds.l.smithi157.stdout:Jun 23 01:18:00 smithi157 systemd[1]: Starting Ceph mds.l for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:18:01.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:01 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/2972691954' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T01:18:01.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:01 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/3615144072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:01.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:01 smithi157 ceph-mon[116945]: mds.? [v2:172.21.15.157:6832/1478747127,v1:172.21.15.157:6833/1478747127] up:boot 2023-06-23T01:18:01.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:01 smithi157 ceph-mon[116945]: fsmap 1 up:standby 2023-06-23T01:18:01.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:01 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-06-23T01:18:01.541 INFO:journalctl@ceph.mds.l.smithi157.stdout:Jun 23 01:18:01 smithi157 ceph-mds[149351]: -- 172.21.15.157:0/2848478903 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d0485329c0 con 0x55d048509000 2023-06-23T01:18:01.541 INFO:journalctl@ceph.mds.l.smithi157.stdout:Jun 23 01:18:01 smithi157 conmon[149327]: starting mds.l at 2023-06-23T01:18:01.542 INFO:journalctl@ceph.mds.l.smithi157.stdout:Jun 23 01:18:01 smithi157 bash[149287]: 4771dd6680c4db6935e7aea2974961cc24513667a6ba33c55dc9f6162d826cae 2023-06-23T01:18:01.760 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:01 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/2972691954' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T01:18:01.760 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:01 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/3615144072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:01.760 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:01 smithi120 ceph-mon[79813]: mds.? [v2:172.21.15.157:6832/1478747127,v1:172.21.15.157:6833/1478747127] up:boot 2023-06-23T01:18:01.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:01 smithi120 ceph-mon[79813]: fsmap 1 up:standby 2023-06-23T01:18:01.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:01 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-06-23T01:18:01.876 INFO:journalctl@ceph.mds.l.smithi157.stdout:Jun 23 01:18:01 smithi157 systemd[1]: Started Ceph mds.l for 9efbfa8e-1162-11ee-9b2b-001a4aab830c. 2023-06-23T01:18:01.943 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:01 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/2972691954' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T01:18:01.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:01 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/3615144072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:01.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:01 smithi134 ceph-mon[116146]: mds.? [v2:172.21.15.157:6832/1478747127,v1:172.21.15.157:6833/1478747127] up:boot 2023-06-23T01:18:01.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:01 smithi134 ceph-mon[116146]: fsmap 1 up:standby 2023-06-23T01:18:01.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:01 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-06-23T01:18:02.876 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:02 smithi157 ceph-mon[116945]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:02.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:02 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:02.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:02 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:02.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:02 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:02.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:02 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T01:18:02.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:02 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T01:18:02.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:02 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:18:02.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:02 smithi157 ceph-mon[116945]: Deploying daemon mds.k on smithi134 2023-06-23T01:18:02.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:02 smithi134 ceph-mon[116146]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:02.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:02 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:02.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:02 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:02.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:02 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:02.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:02 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T01:18:02.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:02 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T01:18:02.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:02 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:18:02.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:02 smithi134 ceph-mon[116146]: Deploying daemon mds.k on smithi134 2023-06-23T01:18:03.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:02 smithi120 ceph-mon[79813]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:03.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:02 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:03.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:02 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:03.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:02 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:03.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:02 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T01:18:03.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:02 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T01:18:03.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:02 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:18:03.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:02 smithi120 ceph-mon[79813]: Deploying daemon mds.k on smithi134 2023-06-23T01:18:03.065 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T01:18:03.258 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.254+0000 7fdfb41ff700 1 -- 172.21.15.120:0/2177094142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfac071910 msgr2=0x7fdfac06fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:03.259 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.254+0000 7fdfb41ff700 1 --2- 172.21.15.120:0/2177094142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfac071910 0x7fdfac06fe20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdfa80049c0 tx=0x7fdfa8009e30 comp rx=0 tx=0).stop 2023-06-23T01:18:03.259 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.254+0000 7fdfb41ff700 1 -- 172.21.15.120:0/2177094142 shutdown_connections 2023-06-23T01:18:03.259 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.254+0000 7fdfb41ff700 1 --2- 172.21.15.120:0/2177094142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfac071910 0x7fdfac06fe20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:03.259 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.254+0000 7fdfb41ff700 1 --2- 172.21.15.120:0/2177094142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfac104c30 0x7fdfac107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:03.259 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.254+0000 7fdfb41ff700 1 --2- 172.21.15.120:0/2177094142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfac070360 0x7fdfac0707d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:03.259 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.254+0000 7fdfb41ff700 1 -- 172.21.15.120:0/2177094142 >> 172.21.15.120:0/2177094142 conn(0x7fdfac0f8450 msgr2=0x7fdfac0fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:03.260 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.254+0000 7fdfb41ff700 1 -- 172.21.15.120:0/2177094142 shutdown_connections 2023-06-23T01:18:03.260 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.254+0000 7fdfb41ff700 1 -- 172.21.15.120:0/2177094142 wait complete. 2023-06-23T01:18:03.261 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.255+0000 7fdfb41ff700 1 Processor -- start 2023-06-23T01:18:03.262 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.255+0000 7fdfb41ff700 1 -- start start 2023-06-23T01:18:03.262 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.255+0000 7fdfb41ff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfac070360 0x7fdfac10fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:03.262 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.255+0000 7fdfb41ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfac071910 0x7fdfac1102c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:03.262 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.255+0000 7fdfb41ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfac104c30 0x7fdfac117310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:03.262 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.256+0000 7fdfb41ff700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac117950 con 0x7fdfac070360 2023-06-23T01:18:03.262 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.256+0000 7fdfb41ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac117ac0 con 0x7fdfac104c30 2023-06-23T01:18:03.262 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.256+0000 7fdfb41ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac117c80 con 0x7fdfac071910 2023-06-23T01:18:03.263 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.256+0000 7fdfb1f9b700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfac070360 0x7fdfac10fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:03.263 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.256+0000 7fdfb1f9b700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfac070360 0x7fdfac10fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60712/0 (socket says 172.21.15.120:60712) 2023-06-23T01:18:03.263 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.256+0000 7fdfb1f9b700 1 -- 172.21.15.120:0/126220927 learned_addr learned my addr 172.21.15.120:0/126220927 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:18:03.263 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.256+0000 7fdfb279c700 1 --2- 172.21.15.120:0/126220927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfac104c30 0x7fdfac117310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:03.263 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.256+0000 7fdfb179a700 1 --2- 172.21.15.120:0/126220927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfac071910 0x7fdfac1102c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:03.263 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.256+0000 7fdfb1f9b700 1 -- 172.21.15.120:0/126220927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfac071910 msgr2=0x7fdfac1102c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:03.263 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.256+0000 7fdfb1f9b700 1 --2- 172.21.15.120:0/126220927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfac071910 0x7fdfac1102c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:03.263 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.256+0000 7fdfb1f9b700 1 -- 172.21.15.120:0/126220927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfac104c30 msgr2=0x7fdfac117310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:03.264 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.256+0000 7fdfb1f9b700 1 --2- 172.21.15.120:0/126220927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfac104c30 0x7fdfac117310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:03.264 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.256+0000 7fdfb1f9b700 1 -- 172.21.15.120:0/126220927 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf9c005040 con 0x7fdfac070360 2023-06-23T01:18:03.264 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.256+0000 7fdfb1f9b700 1 --2- 172.21.15.120:0/126220927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfac070360 0x7fdfac10fd80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8009400 tx=0x7fdfa8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:03.264 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.257+0000 7fdfa27fc700 1 -- 172.21.15.120:0/126220927 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa8020070 con 0x7fdfac070360 2023-06-23T01:18:03.264 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.257+0000 7fdfa27fc700 1 -- 172.21.15.120:0/126220927 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdfa80041a0 con 0x7fdfac070360 2023-06-23T01:18:03.264 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.257+0000 7fdfb41ff700 1 -- 172.21.15.120:0/126220927 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfa8005040 con 0x7fdfac070360 2023-06-23T01:18:03.264 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.257+0000 7fdfa27fc700 1 -- 172.21.15.120:0/126220927 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa800e070 con 0x7fdfac070360 2023-06-23T01:18:03.264 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.257+0000 7fdfb41ff700 1 -- 172.21.15.120:0/126220927 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfac118180 con 0x7fdfac070360 2023-06-23T01:18:03.265 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.258+0000 7fdfb41ff700 1 -- 172.21.15.120:0/126220927 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfac10a7a0 con 0x7fdfac070360 2023-06-23T01:18:03.265 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.259+0000 7fdfa27fc700 1 -- 172.21.15.120:0/126220927 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7fdfa801c430 con 0x7fdfac070360 2023-06-23T01:18:03.265 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.261+0000 7fdfa27fc700 1 --2- 172.21.15.120:0/126220927 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fdf9809b700 0x7fdf9809dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:03.266 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.261+0000 7fdfa27fc700 1 -- 172.21.15.120:0/126220927 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7fdfa80c5a00 con 0x7fdfac070360 2023-06-23T01:18:03.266 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.261+0000 7fdfb179a700 1 --2- 172.21.15.120:0/126220927 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fdf9809b700 0x7fdf9809dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:03.266 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.262+0000 7fdfb179a700 1 --2- 172.21.15.120:0/126220927 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fdf9809b700 0x7fdf9809dbb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c009fd0 tx=0x7fdf9c009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:03.268 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.264+0000 7fdfa27fc700 1 -- 172.21.15.120:0/126220927 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fdfa8091660 con 0x7fdfac070360 2023-06-23T01:18:03.503 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.499+0000 7fdfb41ff700 1 -- 172.21.15.120:0/126220927 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdfac10a930 con 0x7fdfac070360 2023-06-23T01:18:03.504 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.500+0000 7fdfa27fc700 1 -- 172.21.15.120:0/126220927 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1260 (secure 0 0 0) 0x7fdfac10a930 con 0x7fdfac070360 2023-06-23T01:18:03.505 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:18:03.505 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24472,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1478747127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1478747127},{"type":"v1","addr":"172.21.15.157:6833","nonce":1478747127}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-06-23T01:18:03.507 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.502+0000 7fdfb41ff700 1 -- 172.21.15.120:0/126220927 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fdf9809b700 msgr2=0x7fdf9809dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:03.507 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.502+0000 7fdfb41ff700 1 --2- 172.21.15.120:0/126220927 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fdf9809b700 0x7fdf9809dbb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c009fd0 tx=0x7fdf9c009e30 comp rx=0 tx=0).stop 2023-06-23T01:18:03.507 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.502+0000 7fdfb41ff700 1 -- 172.21.15.120:0/126220927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfac070360 msgr2=0x7fdfac10fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:03.507 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.502+0000 7fdfb41ff700 1 --2- 172.21.15.120:0/126220927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfac070360 0x7fdfac10fd80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8009400 tx=0x7fdfa8009e30 comp rx=0 tx=0).stop 2023-06-23T01:18:03.507 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.502+0000 7fdfb41ff700 1 -- 172.21.15.120:0/126220927 shutdown_connections 2023-06-23T01:18:03.508 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.502+0000 7fdfb41ff700 1 --2- 172.21.15.120:0/126220927 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fdf9809b700 0x7fdf9809dbb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:03.508 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.502+0000 7fdfb41ff700 1 --2- 172.21.15.120:0/126220927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfac070360 0x7fdfac10fd80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:03.508 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.502+0000 7fdfb41ff700 1 --2- 172.21.15.120:0/126220927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfac104c30 0x7fdfac117310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:03.508 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.502+0000 7fdfb41ff700 1 --2- 172.21.15.120:0/126220927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfac071910 0x7fdfac1102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:03.508 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.502+0000 7fdfb41ff700 1 -- 172.21.15.120:0/126220927 >> 172.21.15.120:0/126220927 conn(0x7fdfac0f8450 msgr2=0x7fdfac106d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:03.508 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.503+0000 7fdfb41ff700 1 -- 172.21.15.120:0/126220927 shutdown_connections 2023-06-23T01:18:03.508 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:03.503+0000 7fdfb41ff700 1 -- 172.21.15.120:0/126220927 wait complete. 2023-06-23T01:18:03.509 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 2 2023-06-23T01:18:03.680 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:03 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/126220927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:03.680 INFO:journalctl@ceph.mds.k.smithi134.stdout:Jun 23 01:18:03 smithi134 systemd[1]: Starting Ceph mds.k for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:18:03.876 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:03 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/126220927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:04.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:03 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/126220927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:04.194 INFO:journalctl@ceph.mds.k.smithi134.stdout:Jun 23 01:18:03 smithi134 ceph-mds[147257]: -- 172.21.15.134:0/1676759739 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5591a93a29c0 con 0x5591a9378800 2023-06-23T01:18:04.194 INFO:journalctl@ceph.mds.k.smithi134.stdout:Jun 23 01:18:03 smithi134 conmon[147233]: starting mds.k at 2023-06-23T01:18:04.194 INFO:journalctl@ceph.mds.k.smithi134.stdout:Jun 23 01:18:03 smithi134 bash[147193]: 218bd967cacfc317d285e07b24105cc6123e5c8af23e90779d1856745728a8c5 2023-06-23T01:18:04.194 INFO:journalctl@ceph.mds.k.smithi134.stdout:Jun 23 01:18:03 smithi134 systemd[1]: Started Ceph mds.k for 9efbfa8e-1162-11ee-9b2b-001a4aab830c. 2023-06-23T01:18:05.164 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:04 smithi120 ceph-mon[79813]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:05.164 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:04 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:05.164 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:04 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:05.165 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:04 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:05.165 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:04 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T01:18:05.165 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:04 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T01:18:05.165 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:04 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:18:05.165 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:04 smithi120 ceph-mon[79813]: mds.? [v2:172.21.15.134:6832/3214085078,v1:172.21.15.134:6833/3214085078] up:boot 2023-06-23T01:18:05.166 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:04 smithi120 ceph-mon[79813]: fsmap 2 up:standby 2023-06-23T01:18:05.166 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:04 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-06-23T01:18:05.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:04 smithi134 ceph-mon[116146]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:05.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:04 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:05.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:04 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:05.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:04 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:05.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:04 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T01:18:05.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:04 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T01:18:05.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:04 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:18:05.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:04 smithi134 ceph-mon[116146]: mds.? [v2:172.21.15.134:6832/3214085078,v1:172.21.15.134:6833/3214085078] up:boot 2023-06-23T01:18:05.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:04 smithi134 ceph-mon[116146]: fsmap 2 up:standby 2023-06-23T01:18:05.195 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:04 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-06-23T01:18:05.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:04 smithi157 ceph-mon[116945]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:05.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:04 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:05.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:04 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:05.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:04 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:05.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:04 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T01:18:05.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:04 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T01:18:05.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:04 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:18:05.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:04 smithi157 ceph-mon[116945]: mds.? [v2:172.21.15.134:6832/3214085078,v1:172.21.15.134:6833/3214085078] up:boot 2023-06-23T01:18:05.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:04 smithi157 ceph-mon[116945]: fsmap 2 up:standby 2023-06-23T01:18:05.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:04 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-06-23T01:18:05.526 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T01:18:05.923 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.919+0000 7f4b4699d700 1 -- 172.21.15.120:0/1496383244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b40070300 msgr2=0x7f4b40070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:05.924 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.919+0000 7f4b4699d700 1 --2- 172.21.15.120:0/1496383244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b40070300 0x7f4b40070770 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4b30004900 tx=0x7f4b30009e30 comp rx=0 tx=0).stop 2023-06-23T01:18:05.924 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.919+0000 7f4b4699d700 1 -- 172.21.15.120:0/1496383244 shutdown_connections 2023-06-23T01:18:05.924 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.919+0000 7f4b4699d700 1 --2- 172.21.15.120:0/1496383244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b40104a40 0x7f4b40106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:05.924 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.919+0000 7f4b4699d700 1 --2- 172.21.15.120:0/1496383244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b40071950 0x7f4b4006fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:05.924 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.919+0000 7f4b4699d700 1 --2- 172.21.15.120:0/1496383244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b40070300 0x7f4b40070770 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:05.925 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.919+0000 7f4b4699d700 1 -- 172.21.15.120:0/1496383244 >> 172.21.15.120:0/1496383244 conn(0x7f4b400f82a0 msgr2=0x7f4b400fa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:05.925 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.919+0000 7f4b4699d700 1 -- 172.21.15.120:0/1496383244 shutdown_connections 2023-06-23T01:18:05.925 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.919+0000 7f4b4699d700 1 -- 172.21.15.120:0/1496383244 wait complete. 2023-06-23T01:18:05.926 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.920+0000 7f4b4699d700 1 Processor -- start 2023-06-23T01:18:05.927 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.920+0000 7f4b4699d700 1 -- start start 2023-06-23T01:18:05.927 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.920+0000 7f4b4699d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b40070300 0x7f4b4010f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:05.927 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.921+0000 7f4b4699d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b40071950 0x7f4b4010fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:05.927 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.921+0000 7f4b4699d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b40104a40 0x7f4b40116f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:05.927 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.921+0000 7f4b4699d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b40117520 con 0x7f4b40104a40 2023-06-23T01:18:05.927 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.921+0000 7f4b4699d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b40117690 con 0x7f4b40071950 2023-06-23T01:18:05.928 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.921+0000 7f4b4699d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b40117850 con 0x7f4b40070300 2023-06-23T01:18:05.928 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.921+0000 7f4b4619c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b40104a40 0x7f4b40116f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:05.928 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.921+0000 7f4b4619c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b40104a40 0x7f4b40116f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60730/0 (socket says 172.21.15.120:60730) 2023-06-23T01:18:05.928 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.921+0000 7f4b4619c700 1 -- 172.21.15.120:0/3503393044 learned_addr learned my addr 172.21.15.120:0/3503393044 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:18:05.928 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.921+0000 7f4b4599b700 1 --2- 172.21.15.120:0/3503393044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b40070300 0x7f4b4010f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:05.928 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.921+0000 7f4b4519a700 1 --2- 172.21.15.120:0/3503393044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b40071950 0x7f4b4010fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:05.929 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.921+0000 7f4b4619c700 1 -- 172.21.15.120:0/3503393044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b40070300 msgr2=0x7f4b4010f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:05.929 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.921+0000 7f4b4619c700 1 --2- 172.21.15.120:0/3503393044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b40070300 0x7f4b4010f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:05.929 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.921+0000 7f4b4619c700 1 -- 172.21.15.120:0/3503393044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b40071950 msgr2=0x7f4b4010fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:05.929 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.921+0000 7f4b4619c700 1 --2- 172.21.15.120:0/3503393044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b40071950 0x7f4b4010fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:05.929 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.921+0000 7f4b4619c700 1 -- 172.21.15.120:0/3503393044 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b38009940 con 0x7f4b40104a40 2023-06-23T01:18:05.930 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.921+0000 7f4b4619c700 1 --2- 172.21.15.120:0/3503393044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b40104a40 0x7f4b40116f10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4b38001f30 tx=0x7f4b38004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:05.930 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.922+0000 7f4b367fc700 1 -- 172.21.15.120:0/3503393044 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3800fc00 con 0x7f4b40104a40 2023-06-23T01:18:05.930 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.922+0000 7f4b367fc700 1 -- 172.21.15.120:0/3503393044 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b38007df0 con 0x7f4b40104a40 2023-06-23T01:18:05.930 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.922+0000 7f4b4699d700 1 -- 172.21.15.120:0/3503393044 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b30005040 con 0x7f4b40104a40 2023-06-23T01:18:05.930 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.922+0000 7f4b367fc700 1 -- 172.21.15.120:0/3503393044 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b38015470 con 0x7f4b40104a40 2023-06-23T01:18:05.930 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.922+0000 7f4b4699d700 1 -- 172.21.15.120:0/3503393044 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b40117ec0 con 0x7f4b40104a40 2023-06-23T01:18:05.931 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.923+0000 7f4b4699d700 1 -- 172.21.15.120:0/3503393044 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b40103190 con 0x7f4b40104a40 2023-06-23T01:18:05.931 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.925+0000 7f4b367fc700 1 -- 172.21.15.120:0/3503393044 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7f4b38007920 con 0x7f4b40104a40 2023-06-23T01:18:05.932 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.926+0000 7f4b367fc700 1 --2- 172.21.15.120:0/3503393044 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f4b2c09b680 0x7f4b2c09db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:05.932 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.926+0000 7f4b367fc700 1 -- 172.21.15.120:0/3503393044 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f4b380c4800 con 0x7f4b40104a40 2023-06-23T01:18:05.932 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.927+0000 7f4b4599b700 1 --2- 172.21.15.120:0/3503393044 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f4b2c09b680 0x7f4b2c09db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:05.932 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.927+0000 7f4b4599b700 1 --2- 172.21.15.120:0/3503393044 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f4b2c09b680 0x7f4b2c09db30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c009b80 tx=0x7f4b3c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:05.934 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:05.930+0000 7f4b367fc700 1 -- 172.21.15.120:0/3503393044 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f4b38090460 con 0x7f4b40104a40 2023-06-23T01:18:06.116 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:05 smithi120 ceph-mon[79813]: Deploying daemon mds.j on smithi120 2023-06-23T01:18:06.193 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:05 smithi134 ceph-mon[116146]: Deploying daemon mds.j on smithi120 2023-06-23T01:18:06.199 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:06.195+0000 7f4b4699d700 1 -- 172.21.15.120:0/3503393044 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b4002b120 con 0x7f4b40104a40 2023-06-23T01:18:06.200 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:06.196+0000 7f4b367fc700 1 -- 172.21.15.120:0/3503393044 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2020 (secure 0 0 0) 0x7f4b38093a80 con 0x7f4b40104a40 2023-06-23T01:18:06.200 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:18:06.200 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3214085078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3214085078},{"type":"v1","addr":"172.21.15.134:6833","nonce":3214085078}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1478747127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1478747127},{"type":"v1","addr":"172.21.15.157:6833","nonce":1478747127}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-06-23T01:18:06.203 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:06.199+0000 7f4b4699d700 1 -- 172.21.15.120:0/3503393044 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f4b2c09b680 msgr2=0x7f4b2c09db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:06.203 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:06.199+0000 7f4b4699d700 1 --2- 172.21.15.120:0/3503393044 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f4b2c09b680 0x7f4b2c09db30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c009b80 tx=0x7f4b3c00b040 comp rx=0 tx=0).stop 2023-06-23T01:18:06.203 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:06.199+0000 7f4b4699d700 1 -- 172.21.15.120:0/3503393044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b40104a40 msgr2=0x7f4b40116f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:06.203 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:06.199+0000 7f4b4699d700 1 --2- 172.21.15.120:0/3503393044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b40104a40 0x7f4b40116f10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4b38001f30 tx=0x7f4b38004d10 comp rx=0 tx=0).stop 2023-06-23T01:18:06.204 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:06.199+0000 7f4b4699d700 1 -- 172.21.15.120:0/3503393044 shutdown_connections 2023-06-23T01:18:06.204 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:06.199+0000 7f4b4699d700 1 --2- 172.21.15.120:0/3503393044 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f4b2c09b680 0x7f4b2c09db30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:06.204 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:06.199+0000 7f4b4699d700 1 --2- 172.21.15.120:0/3503393044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b40104a40 0x7f4b40116f10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:06.205 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:06.199+0000 7f4b4699d700 1 --2- 172.21.15.120:0/3503393044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b40070300 0x7f4b4010f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:06.205 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:06.199+0000 7f4b4699d700 1 --2- 172.21.15.120:0/3503393044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b40071950 0x7f4b4010fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:06.205 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:06.199+0000 7f4b4699d700 1 -- 172.21.15.120:0/3503393044 >> 172.21.15.120:0/3503393044 conn(0x7f4b400f82a0 msgr2=0x7f4b40106b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:06.206 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:06.199+0000 7f4b4699d700 1 -- 172.21.15.120:0/3503393044 shutdown_connections 2023-06-23T01:18:06.206 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:06.199+0000 7f4b4699d700 1 -- 172.21.15.120:0/3503393044 wait complete. 2023-06-23T01:18:06.207 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 3 2023-06-23T01:18:06.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:05 smithi157 ceph-mon[116945]: Deploying daemon mds.j on smithi120 2023-06-23T01:18:06.510 INFO:journalctl@ceph.mds.j.smithi120.stdout:Jun 23 01:18:06 smithi120 systemd[1]: Starting Ceph mds.j for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:18:07.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:06 smithi134 ceph-mon[116146]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:07.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:06 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/3503393044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:07.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:06 smithi120 ceph-mon[79813]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:07.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:06 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/3503393044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:07.261 INFO:journalctl@ceph.mds.j.smithi120.stdout:Jun 23 01:18:06 smithi120 bash[122594]: ce547800daa38de6ebb40e691b4cca8db7785693c9271aaab69e1fe4f61005a9 2023-06-23T01:18:07.261 INFO:journalctl@ceph.mds.j.smithi120.stdout:Jun 23 01:18:06 smithi120 ceph-mds[122658]: -- 172.21.15.120:0/382553562 <== mon.0 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ed2df309c0 con 0x55ed2df07000 2023-06-23T01:18:07.261 INFO:journalctl@ceph.mds.j.smithi120.stdout:Jun 23 01:18:06 smithi120 conmon[122634]: starting mds.j at 2023-06-23T01:18:07.261 INFO:journalctl@ceph.mds.j.smithi120.stdout:Jun 23 01:18:06 smithi120 systemd[1]: Started Ceph mds.j for 9efbfa8e-1162-11ee-9b2b-001a4aab830c. 2023-06-23T01:18:07.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:06 smithi157 ceph-mon[116945]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:07.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:06 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/3503393044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:08.223 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T01:18:08.285 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:08 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:08.286 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:08 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:08.286 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:08 smithi120 ceph-mon[79813]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:08.286 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:08 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:08.286 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:08 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T01:18:08.286 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:08 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T01:18:08.287 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:08 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:18:08.287 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:08 smithi120 ceph-mon[79813]: Deploying daemon mds.d on smithi120 2023-06-23T01:18:08.287 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:08 smithi120 ceph-mon[79813]: mds.? [v2:172.21.15.120:6834/1538161586,v1:172.21.15.120:6835/1538161586] up:boot 2023-06-23T01:18:08.287 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:08 smithi120 ceph-mon[79813]: fsmap 3 up:standby 2023-06-23T01:18:08.287 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:08 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-06-23T01:18:08.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:08 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:08.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:08 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:08.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:08 smithi157 ceph-mon[116945]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:08.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:08 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:08.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:08 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T01:18:08.379 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:08 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T01:18:08.379 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:08 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:18:08.379 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:08 smithi157 ceph-mon[116945]: Deploying daemon mds.d on smithi120 2023-06-23T01:18:08.379 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:08 smithi157 ceph-mon[116945]: mds.? [v2:172.21.15.120:6834/1538161586,v1:172.21.15.120:6835/1538161586] up:boot 2023-06-23T01:18:08.380 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:08 smithi157 ceph-mon[116945]: fsmap 3 up:standby 2023-06-23T01:18:08.380 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:08 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-06-23T01:18:08.422 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.417+0000 7f69e359d700 1 -- 172.21.15.120:0/1721302426 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f69dc0fdd20 msgr2=0x7f69dc0fe170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:08.422 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.417+0000 7f69e359d700 1 --2- 172.21.15.120:0/1721302426 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f69dc0fdd20 0x7f69dc0fe170 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f69cc0049c0 tx=0x7f69cc009e30 comp rx=0 tx=0).stop 2023-06-23T01:18:08.422 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.418+0000 7f69e359d700 1 -- 172.21.15.120:0/1721302426 shutdown_connections 2023-06-23T01:18:08.423 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.418+0000 7f69e359d700 1 --2- 172.21.15.120:0/1721302426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69dc0fe6b0 0x7f69dc104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:08.423 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.418+0000 7f69e359d700 1 --2- 172.21.15.120:0/1721302426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69dc0fcb10 0x7f69dc0fcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:08.423 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.418+0000 7f69e359d700 1 --2- 172.21.15.120:0/1721302426 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f69dc0fdd20 0x7f69dc0fe170 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:08.423 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.418+0000 7f69e359d700 1 -- 172.21.15.120:0/1721302426 >> 172.21.15.120:0/1721302426 conn(0x7f69dc0f8270 msgr2=0x7f69dc0fa6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:08.423 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.418+0000 7f69e359d700 1 -- 172.21.15.120:0/1721302426 shutdown_connections 2023-06-23T01:18:08.423 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.418+0000 7f69e359d700 1 -- 172.21.15.120:0/1721302426 wait complete. 2023-06-23T01:18:08.426 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.418+0000 7f69e359d700 1 Processor -- start 2023-06-23T01:18:08.426 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.419+0000 7f69e359d700 1 -- start start 2023-06-23T01:18:08.426 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.419+0000 7f69e359d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f69dc0fcb10 0x7f69dc071720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:08.426 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.419+0000 7f69e359d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69dc0fdd20 0x7f69dc06fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:08.426 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.419+0000 7f69e359d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69dc0fe6b0 0x7f69dc0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:08.426 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.419+0000 7f69e359d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69dc070a30 con 0x7f69dc0fcb10 2023-06-23T01:18:08.427 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.419+0000 7f69e359d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69dc070ba0 con 0x7f69dc0fe6b0 2023-06-23T01:18:08.427 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.419+0000 7f69e359d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69dc117800 con 0x7f69dc0fdd20 2023-06-23T01:18:08.427 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.419+0000 7f69e259b700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f69dc0fcb10 0x7f69dc071720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:08.427 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.419+0000 7f69e259b700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f69dc0fcb10 0x7f69dc071720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60762/0 (socket says 172.21.15.120:60762) 2023-06-23T01:18:08.427 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.419+0000 7f69e259b700 1 -- 172.21.15.120:0/4130067331 learned_addr learned my addr 172.21.15.120:0/4130067331 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:18:08.427 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.419+0000 7f69e2d9c700 1 --2- 172.21.15.120:0/4130067331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69dc0fe6b0 0x7f69dc0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:08.427 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.419+0000 7f69e1d9a700 1 --2- 172.21.15.120:0/4130067331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69dc0fdd20 0x7f69dc06fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:08.427 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.420+0000 7f69e259b700 1 -- 172.21.15.120:0/4130067331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69dc0fdd20 msgr2=0x7f69dc06fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:08.428 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.420+0000 7f69e259b700 1 --2- 172.21.15.120:0/4130067331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69dc0fdd20 0x7f69dc06fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:08.428 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.420+0000 7f69e259b700 1 -- 172.21.15.120:0/4130067331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69dc0fe6b0 msgr2=0x7f69dc0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:08.428 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.420+0000 7f69e259b700 1 --2- 172.21.15.120:0/4130067331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69dc0fe6b0 0x7f69dc0702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:08.428 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.420+0000 7f69e259b700 1 -- 172.21.15.120:0/4130067331 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d8005040 con 0x7f69dc0fcb10 2023-06-23T01:18:08.428 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.420+0000 7f69e259b700 1 --2- 172.21.15.120:0/4130067331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f69dc0fcb10 0x7f69dc071720 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f69d8009750 tx=0x7f69d8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:08.428 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.420+0000 7f69d2ffd700 1 -- 172.21.15.120:0/4130067331 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d800e7e0 con 0x7f69dc0fcb10 2023-06-23T01:18:08.428 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.420+0000 7f69d2ffd700 1 -- 172.21.15.120:0/4130067331 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69d800f040 con 0x7f69dc0fcb10 2023-06-23T01:18:08.429 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.420+0000 7f69e359d700 1 -- 172.21.15.120:0/4130067331 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69cc005040 con 0x7f69dc0fcb10 2023-06-23T01:18:08.429 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.420+0000 7f69d2ffd700 1 -- 172.21.15.120:0/4130067331 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d8013e40 con 0x7f69dc0fcb10 2023-06-23T01:18:08.429 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.420+0000 7f69e359d700 1 -- 172.21.15.120:0/4130067331 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69dc117e80 con 0x7f69dc0fcb10 2023-06-23T01:18:08.429 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.421+0000 7f69e359d700 1 -- 172.21.15.120:0/4130067331 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69dc108330 con 0x7f69dc0fcb10 2023-06-23T01:18:08.430 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.423+0000 7f69d2ffd700 1 -- 172.21.15.120:0/4130067331 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7f69d8013490 con 0x7f69dc0fcb10 2023-06-23T01:18:08.430 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.424+0000 7f69d2ffd700 1 --2- 172.21.15.120:0/4130067331 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f69c809b700 0x7f69c809dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:08.430 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.425+0000 7f69d2ffd700 1 -- 172.21.15.120:0/4130067331 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f69d80c4830 con 0x7f69dc0fcb10 2023-06-23T01:18:08.430 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.425+0000 7f69e1d9a700 1 --2- 172.21.15.120:0/4130067331 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f69c809b700 0x7f69c809dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:08.430 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.425+0000 7f69e1d9a700 1 --2- 172.21.15.120:0/4130067331 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f69c809b700 0x7f69c809dbb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f69cc007f50 tx=0x7f69cc009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:08.432 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.428+0000 7f69d2ffd700 1 -- 172.21.15.120:0/4130067331 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f69d8090490 con 0x7f69dc0fcb10 2023-06-23T01:18:08.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:08 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:08.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:08 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:08.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:08 smithi134 ceph-mon[116146]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:08.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:08 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:08.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:08 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T01:18:08.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:08 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T01:18:08.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:08 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:18:08.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:08 smithi134 ceph-mon[116146]: Deploying daemon mds.d on smithi120 2023-06-23T01:18:08.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:08 smithi134 ceph-mon[116146]: mds.? [v2:172.21.15.120:6834/1538161586,v1:172.21.15.120:6835/1538161586] up:boot 2023-06-23T01:18:08.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:08 smithi134 ceph-mon[116146]: fsmap 3 up:standby 2023-06-23T01:18:08.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:08 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-06-23T01:18:08.715 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.711+0000 7f69e359d700 1 -- 172.21.15.120:0/4130067331 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69dc103950 con 0x7f69dc0fcb10 2023-06-23T01:18:08.717 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.713+0000 7f69d2ffd700 1 -- 172.21.15.120:0/4130067331 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2780 (secure 0 0 0) 0x7f69d8093ab0 con 0x7f69dc0fcb10 2023-06-23T01:18:08.717 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:18:08.717 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14427,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1538161586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1538161586},{"type":"v1","addr":"172.21.15.120:6835","nonce":1538161586}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3214085078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3214085078},{"type":"v1","addr":"172.21.15.134:6833","nonce":3214085078}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1478747127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1478747127},{"type":"v1","addr":"172.21.15.157:6833","nonce":1478747127}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-06-23T01:18:08.720 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.715+0000 7f69e359d700 1 -- 172.21.15.120:0/4130067331 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f69c809b700 msgr2=0x7f69c809dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:08.720 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.715+0000 7f69e359d700 1 --2- 172.21.15.120:0/4130067331 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f69c809b700 0x7f69c809dbb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f69cc007f50 tx=0x7f69cc009e30 comp rx=0 tx=0).stop 2023-06-23T01:18:08.720 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.715+0000 7f69e359d700 1 -- 172.21.15.120:0/4130067331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f69dc0fcb10 msgr2=0x7f69dc071720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:08.721 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.715+0000 7f69e359d700 1 --2- 172.21.15.120:0/4130067331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f69dc0fcb10 0x7f69dc071720 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f69d8009750 tx=0x7f69d8009e30 comp rx=0 tx=0).stop 2023-06-23T01:18:08.721 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.715+0000 7f69e359d700 1 -- 172.21.15.120:0/4130067331 shutdown_connections 2023-06-23T01:18:08.721 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.715+0000 7f69e359d700 1 --2- 172.21.15.120:0/4130067331 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f69c809b700 0x7f69c809dbb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:08.723 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.715+0000 7f69e359d700 1 --2- 172.21.15.120:0/4130067331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f69dc0fcb10 0x7f69dc071720 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:08.723 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.715+0000 7f69e359d700 1 --2- 172.21.15.120:0/4130067331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69dc0fe6b0 0x7f69dc0702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:08.723 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.715+0000 7f69e359d700 1 --2- 172.21.15.120:0/4130067331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69dc0fdd20 0x7f69dc06fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:08.723 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.715+0000 7f69e359d700 1 -- 172.21.15.120:0/4130067331 >> 172.21.15.120:0/4130067331 conn(0x7f69dc0f8270 msgr2=0x7f69dc0fa3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:08.724 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.716+0000 7f69e359d700 1 -- 172.21.15.120:0/4130067331 shutdown_connections 2023-06-23T01:18:08.724 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:08.716+0000 7f69e359d700 1 -- 172.21.15.120:0/4130067331 wait complete. 2023-06-23T01:18:08.724 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 4 2023-06-23T01:18:09.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:09 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/4130067331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:09.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:09 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/4130067331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:09.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:09 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/4130067331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:09.511 INFO:journalctl@ceph.mds.d.smithi120.stdout:Jun 23 01:18:09 smithi120 systemd[1]: Starting Ceph mds.d for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:18:10.261 INFO:journalctl@ceph.mds.d.smithi120.stdout:Jun 23 01:18:09 smithi120 bash[123196]: 76ffe92ba9bf68d9e29696b155871aee436e55cfc857985ca6e22fd236dacbfb 2023-06-23T01:18:10.261 INFO:journalctl@ceph.mds.d.smithi120.stdout:Jun 23 01:18:09 smithi120 ceph-mds[123260]: -- 172.21.15.120:0/840858357 <== mon.0 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ce676089c0 con 0x55ce675df000 2023-06-23T01:18:10.261 INFO:journalctl@ceph.mds.d.smithi120.stdout:Jun 23 01:18:09 smithi120 conmon[123236]: starting mds.d at 2023-06-23T01:18:10.262 INFO:journalctl@ceph.mds.d.smithi120.stdout:Jun 23 01:18:09 smithi120 systemd[1]: Started Ceph mds.d for 9efbfa8e-1162-11ee-9b2b-001a4aab830c. 2023-06-23T01:18:10.739 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T01:18:10.929 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.924+0000 7f9306bff700 1 -- 172.21.15.120:0/2479291577 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93000fdf80 msgr2=0x7f93000fe3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:10.929 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.924+0000 7f9306bff700 1 --2- 172.21.15.120:0/2479291577 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93000fdf80 0x7f93000fe3f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f92ec0049c0 tx=0x7f92ec009e30 comp rx=0 tx=0).stop 2023-06-23T01:18:10.929 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.925+0000 7f9306bff700 1 -- 172.21.15.120:0/2479291577 shutdown_connections 2023-06-23T01:18:10.930 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.925+0000 7f9306bff700 1 --2- 172.21.15.120:0/2479291577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93000fe930 0x7f93001051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:10.930 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.925+0000 7f9306bff700 1 --2- 172.21.15.120:0/2479291577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93000fccd0 0x7f93000fd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:10.930 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.925+0000 7f9306bff700 1 --2- 172.21.15.120:0/2479291577 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93000fdf80 0x7f93000fe3f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:10.930 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.925+0000 7f9306bff700 1 -- 172.21.15.120:0/2479291577 >> 172.21.15.120:0/2479291577 conn(0x7f93000f83f0 msgr2=0x7f93000fa860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:10.930 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.925+0000 7f9306bff700 1 -- 172.21.15.120:0/2479291577 shutdown_connections 2023-06-23T01:18:10.930 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.925+0000 7f9306bff700 1 -- 172.21.15.120:0/2479291577 wait complete. 2023-06-23T01:18:10.930 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.925+0000 7f9306bff700 1 Processor -- start 2023-06-23T01:18:10.932 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.925+0000 7f9306bff700 1 -- start start 2023-06-23T01:18:10.932 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.926+0000 7f9306bff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93000fccd0 0x7f9300071720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:10.932 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.926+0000 7f9306bff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93000fdf80 0x7f930006fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:10.933 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.926+0000 7f9306bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93000fe930 0x7f93000702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:10.933 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.926+0000 7f9306bff700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9300070a30 con 0x7f93000fdf80 2023-06-23T01:18:10.933 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.926+0000 7f9306bff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9300070ba0 con 0x7f93000fccd0 2023-06-23T01:18:10.933 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.926+0000 7f9306bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93001179e0 con 0x7f93000fe930 2023-06-23T01:18:10.933 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.926+0000 7f930499b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93000fccd0 0x7f9300071720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:10.933 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.926+0000 7f930499b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93000fccd0 0x7f9300071720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:42366/0 (socket says 172.21.15.120:42366) 2023-06-23T01:18:10.933 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.926+0000 7f930499b700 1 -- 172.21.15.120:0/2884728513 learned_addr learned my addr 172.21.15.120:0/2884728513 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:18:10.934 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.926+0000 7f92fffff700 1 --2- 172.21.15.120:0/2884728513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93000fdf80 0x7f930006fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:10.934 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.926+0000 7f930519c700 1 --2- 172.21.15.120:0/2884728513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93000fe930 0x7f93000702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:10.934 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.926+0000 7f930499b700 1 -- 172.21.15.120:0/2884728513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93000fe930 msgr2=0x7f93000702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:10.934 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.926+0000 7f930499b700 1 --2- 172.21.15.120:0/2884728513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93000fe930 0x7f93000702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:10.935 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.926+0000 7f930499b700 1 -- 172.21.15.120:0/2884728513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93000fdf80 msgr2=0x7f930006fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:10.935 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.926+0000 7f930499b700 1 --2- 172.21.15.120:0/2884728513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93000fdf80 0x7f930006fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:10.935 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.926+0000 7f930499b700 1 -- 172.21.15.120:0/2884728513 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ec005040 con 0x7f93000fccd0 2023-06-23T01:18:10.936 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.927+0000 7f930499b700 1 --2- 172.21.15.120:0/2884728513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93000fccd0 0x7f9300071720 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f92f4002ad0 tx=0x7f92f40046b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:10.936 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.927+0000 7f92fd7fa700 1 -- 172.21.15.120:0/2884728513 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f4007cc0 con 0x7f93000fccd0 2023-06-23T01:18:10.936 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.927+0000 7f92fd7fa700 1 -- 172.21.15.120:0/2884728513 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92f400f040 con 0x7f93000fccd0 2023-06-23T01:18:10.936 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.927+0000 7f9306bff700 1 -- 172.21.15.120:0/2884728513 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9300117c90 con 0x7f93000fccd0 2023-06-23T01:18:10.937 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.927+0000 7f92fd7fa700 1 -- 172.21.15.120:0/2884728513 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f400d9a0 con 0x7f93000fccd0 2023-06-23T01:18:10.937 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.927+0000 7f9306bff700 1 -- 172.21.15.120:0/2884728513 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9300118180 con 0x7f93000fccd0 2023-06-23T01:18:10.937 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.928+0000 7f9306bff700 1 -- 172.21.15.120:0/2884728513 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93001084e0 con 0x7f93000fccd0 2023-06-23T01:18:10.939 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.930+0000 7f92fd7fa700 1 -- 172.21.15.120:0/2884728513 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7f92f4042bb0 con 0x7f93000fccd0 2023-06-23T01:18:10.939 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.931+0000 7f92fd7fa700 1 --2- 172.21.15.120:0/2884728513 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f92f009b680 0x7f92f009db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:10.939 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.931+0000 7f92fd7fa700 1 -- 172.21.15.120:0/2884728513 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f92f40c4670 con 0x7f93000fccd0 2023-06-23T01:18:10.940 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.931+0000 7f92fffff700 1 --2- 172.21.15.120:0/2884728513 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f92f009b680 0x7f92f009db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:10.940 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.932+0000 7f92fffff700 1 --2- 172.21.15.120:0/2884728513 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f92f009b680 0x7f92f009db30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f92ec009b30 tx=0x7f92ec009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:10.941 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:10.933+0000 7f92fd7fa700 1 -- 172.21.15.120:0/2884728513 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f92f40902d0 con 0x7f93000fccd0 2023-06-23T01:18:11.174 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:11.170+0000 7f9306bff700 1 -- 172.21.15.120:0/2884728513 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9300118570 con 0x7f93000fccd0 2023-06-23T01:18:11.175 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:11.171+0000 7f92fd7fa700 1 -- 172.21.15.120:0/2884728513 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3540 (secure 0 0 0) 0x7f92f4042e60 con 0x7f93000fccd0 2023-06-23T01:18:11.175 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:18:11.175 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14427,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1538161586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1538161586},{"type":"v1","addr":"172.21.15.120:6835","nonce":1538161586}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/2588400819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":2588400819},{"type":"v1","addr":"172.21.15.120:6837","nonce":2588400819}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3214085078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3214085078},{"type":"v1","addr":"172.21.15.134:6833","nonce":3214085078}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1478747127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1478747127},{"type":"v1","addr":"172.21.15.157:6833","nonce":1478747127}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-06-23T01:18:11.177 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:11.173+0000 7f9306bff700 1 -- 172.21.15.120:0/2884728513 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f92f009b680 msgr2=0x7f92f009db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:11.178 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:11.173+0000 7f9306bff700 1 --2- 172.21.15.120:0/2884728513 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f92f009b680 0x7f92f009db30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f92ec009b30 tx=0x7f92ec009e30 comp rx=0 tx=0).stop 2023-06-23T01:18:11.178 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:11.173+0000 7f9306bff700 1 -- 172.21.15.120:0/2884728513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93000fccd0 msgr2=0x7f9300071720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:11.178 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:11.173+0000 7f9306bff700 1 --2- 172.21.15.120:0/2884728513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93000fccd0 0x7f9300071720 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f92f4002ad0 tx=0x7f92f40046b0 comp rx=0 tx=0).stop 2023-06-23T01:18:11.178 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:11.174+0000 7f9306bff700 1 -- 172.21.15.120:0/2884728513 shutdown_connections 2023-06-23T01:18:11.178 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:11.174+0000 7f9306bff700 1 --2- 172.21.15.120:0/2884728513 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f92f009b680 0x7f92f009db30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:11.178 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:11.174+0000 7f9306bff700 1 --2- 172.21.15.120:0/2884728513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93000fe930 0x7f93000702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:11.179 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:11.174+0000 7f9306bff700 1 --2- 172.21.15.120:0/2884728513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93000fccd0 0x7f9300071720 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:11.179 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:11.174+0000 7f9306bff700 1 --2- 172.21.15.120:0/2884728513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93000fdf80 0x7f930006fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:11.179 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:11.174+0000 7f9306bff700 1 -- 172.21.15.120:0/2884728513 >> 172.21.15.120:0/2884728513 conn(0x7f93000f83f0 msgr2=0x7f93000fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:11.179 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:11.174+0000 7f9306bff700 1 -- 172.21.15.120:0/2884728513 shutdown_connections 2023-06-23T01:18:11.179 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:11.174+0000 7f9306bff700 1 -- 172.21.15.120:0/2884728513 wait complete. 2023-06-23T01:18:11.180 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 5 2023-06-23T01:18:11.193 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:10 smithi120 ceph-mon[79813]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:11.194 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:10 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:11.194 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:10 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:11.194 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:10 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:11.194 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:10 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T01:18:11.194 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:10 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T01:18:11.194 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:10 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:18:11.194 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:10 smithi120 ceph-mon[79813]: mds.? [v2:172.21.15.120:6836/2588400819,v1:172.21.15.120:6837/2588400819] up:boot 2023-06-23T01:18:11.195 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:10 smithi120 ceph-mon[79813]: fsmap 4 up:standby 2023-06-23T01:18:11.195 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:10 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-06-23T01:18:11.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:10 smithi157 ceph-mon[116945]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:11.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:10 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:11.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:10 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:11.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:10 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:11.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:10 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T01:18:11.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:10 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T01:18:11.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:10 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:18:11.379 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:10 smithi157 ceph-mon[116945]: mds.? [v2:172.21.15.120:6836/2588400819,v1:172.21.15.120:6837/2588400819] up:boot 2023-06-23T01:18:11.379 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:10 smithi157 ceph-mon[116945]: fsmap 4 up:standby 2023-06-23T01:18:11.379 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:10 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-06-23T01:18:11.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:10 smithi134 ceph-mon[116146]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:11.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:10 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:11.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:10 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:11.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:10 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:11.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:10 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T01:18:11.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:10 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T01:18:11.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:10 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:18:11.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:10 smithi134 ceph-mon[116146]: mds.? [v2:172.21.15.120:6836/2588400819,v1:172.21.15.120:6837/2588400819] up:boot 2023-06-23T01:18:11.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:10 smithi134 ceph-mon[116146]: fsmap 4 up:standby 2023-06-23T01:18:11.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:10 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-06-23T01:18:11.888 INFO:journalctl@ceph.mds.e.smithi134.stdout:Jun 23 01:18:11 smithi134 systemd[1]: Starting Ceph mds.e for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:18:12.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:11 smithi134 ceph-mon[116146]: Deploying daemon mds.e on smithi134 2023-06-23T01:18:12.194 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:11 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/2884728513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:12.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:11 smithi120 ceph-mon[79813]: Deploying daemon mds.e on smithi134 2023-06-23T01:18:12.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:11 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/2884728513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:12.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:11 smithi157 ceph-mon[116945]: Deploying daemon mds.e on smithi134 2023-06-23T01:18:12.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:11 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/2884728513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:12.495 INFO:journalctl@ceph.mds.e.smithi134.stdout:Jun 23 01:18:12 smithi134 ceph-mds[147818]: -- 172.21.15.134:0/759884693 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5634e9fac9c0 con 0x5634e9f82c00 2023-06-23T01:18:12.495 INFO:journalctl@ceph.mds.e.smithi134.stdout:Jun 23 01:18:12 smithi134 conmon[147794]: starting mds.e at 2023-06-23T01:18:12.495 INFO:journalctl@ceph.mds.e.smithi134.stdout:Jun 23 01:18:12 smithi134 bash[147753]: 13778a74dcfbe2294ec0875805a4c57a5fbeb052914fb8be3c22a3d1623b0e79 2023-06-23T01:18:12.943 INFO:journalctl@ceph.mds.e.smithi134.stdout:Jun 23 01:18:12 smithi134 systemd[1]: Started Ceph mds.e for 9efbfa8e-1162-11ee-9b2b-001a4aab830c. 2023-06-23T01:18:13.197 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T01:18:13.256 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:13 smithi120 ceph-mon[79813]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:13.256 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:13 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:13.256 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:13 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:13.256 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:13 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:13.256 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:13 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T01:18:13.257 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:13 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T01:18:13.257 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:13 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:18:13.259 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:13 smithi157 ceph-mon[116945]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:13.259 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:13 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:13.259 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:13 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:13.259 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:13 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:13.259 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:13 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T01:18:13.259 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:13 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T01:18:13.259 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:13 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:18:13.392 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.386+0000 7f48ced9d700 1 -- 172.21.15.120:0/1852781818 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48c8102000 msgr2=0x7f48c81043e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:13.393 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.386+0000 7f48ced9d700 1 --2- 172.21.15.120:0/1852781818 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48c8102000 0x7f48c81043e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f48b80049c0 tx=0x7f48b8009e30 comp rx=0 tx=0).stop 2023-06-23T01:18:13.393 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.386+0000 7f48ced9d700 1 -- 172.21.15.120:0/1852781818 shutdown_connections 2023-06-23T01:18:13.393 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.386+0000 7f48ced9d700 1 --2- 172.21.15.120:0/1852781818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48c8104970 0x7f48c8106e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:13.393 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.386+0000 7f48ced9d700 1 --2- 172.21.15.120:0/1852781818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48c8066bb0 0x7f48c8101ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:13.393 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.386+0000 7f48ced9d700 1 --2- 172.21.15.120:0/1852781818 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48c8102000 0x7f48c81043e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:13.393 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.386+0000 7f48ced9d700 1 -- 172.21.15.120:0/1852781818 >> 172.21.15.120:0/1852781818 conn(0x7f48c80f7130 msgr2=0x7f48c80f9560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:13.393 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.387+0000 7f48ced9d700 1 -- 172.21.15.120:0/1852781818 shutdown_connections 2023-06-23T01:18:13.394 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.387+0000 7f48ced9d700 1 -- 172.21.15.120:0/1852781818 wait complete. 2023-06-23T01:18:13.394 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.387+0000 7f48ced9d700 1 Processor -- start 2023-06-23T01:18:13.394 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.387+0000 7f48ced9d700 1 -- start start 2023-06-23T01:18:13.394 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.388+0000 7f48ced9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48c8066bb0 0x7f48c810f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:13.394 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.388+0000 7f48ced9d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48c8102000 0x7f48c810fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:13.394 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.388+0000 7f48ced9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48c8104970 0x7f48c8116f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:13.394 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.388+0000 7f48ced9d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c8117540 con 0x7f48c8102000 2023-06-23T01:18:13.395 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.388+0000 7f48ced9d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c81176b0 con 0x7f48c8066bb0 2023-06-23T01:18:13.395 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.388+0000 7f48ced9d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c8117870 con 0x7f48c8104970 2023-06-23T01:18:13.395 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.388+0000 7f48cd59a700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48c8102000 0x7f48c810fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:13.395 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.388+0000 7f48ce59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48c8104970 0x7f48c8116f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:13.395 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.388+0000 7f48cd59a700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48c8102000 0x7f48c810fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38038/0 (socket says 172.21.15.120:38038) 2023-06-23T01:18:13.395 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.388+0000 7f48cd59a700 1 -- 172.21.15.120:0/28684971 learned_addr learned my addr 172.21.15.120:0/28684971 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:18:13.395 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.388+0000 7f48cdd9b700 1 --2- 172.21.15.120:0/28684971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48c8066bb0 0x7f48c810f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:13.396 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.388+0000 7f48cd59a700 1 -- 172.21.15.120:0/28684971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48c8104970 msgr2=0x7f48c8116f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:13.396 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.388+0000 7f48cd59a700 1 --2- 172.21.15.120:0/28684971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48c8104970 0x7f48c8116f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:13.396 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.388+0000 7f48cd59a700 1 -- 172.21.15.120:0/28684971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48c8066bb0 msgr2=0x7f48c810f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:13.396 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.388+0000 7f48cd59a700 1 --2- 172.21.15.120:0/28684971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48c8066bb0 0x7f48c810f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:13.396 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.388+0000 7f48cd59a700 1 -- 172.21.15.120:0/28684971 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b8005040 con 0x7f48c8102000 2023-06-23T01:18:13.396 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.388+0000 7f48cd59a700 1 --2- 172.21.15.120:0/28684971 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48c8102000 0x7f48c810fea0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f48b800b3b0 tx=0x7f48b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:13.398 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.389+0000 7f48be7fc700 1 -- 172.21.15.120:0/28684971 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b800e040 con 0x7f48c8102000 2023-06-23T01:18:13.398 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.389+0000 7f48be7fc700 1 -- 172.21.15.120:0/28684971 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48b8009670 con 0x7f48c8102000 2023-06-23T01:18:13.398 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.389+0000 7f48be7fc700 1 -- 172.21.15.120:0/28684971 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b8013cd0 con 0x7f48c8102000 2023-06-23T01:18:13.398 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.389+0000 7f48ced9d700 1 -- 172.21.15.120:0/28684971 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48c8117af0 con 0x7f48c8102000 2023-06-23T01:18:13.398 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.389+0000 7f48ced9d700 1 -- 172.21.15.120:0/28684971 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c8117fd0 con 0x7f48c8102000 2023-06-23T01:18:13.399 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.390+0000 7f48ced9d700 1 -- 172.21.15.120:0/28684971 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48c80f8c60 con 0x7f48c8102000 2023-06-23T01:18:13.399 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.392+0000 7f48be7fc700 1 -- 172.21.15.120:0/28684971 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7f48b8013420 con 0x7f48c8102000 2023-06-23T01:18:13.399 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.393+0000 7f48be7fc700 1 --2- 172.21.15.120:0/28684971 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f48b409b700 0x7f48b409dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:13.399 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.393+0000 7f48be7fc700 1 -- 172.21.15.120:0/28684971 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f48b80c53d0 con 0x7f48c8102000 2023-06-23T01:18:13.399 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.393+0000 7f48cdd9b700 1 --2- 172.21.15.120:0/28684971 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f48b409b700 0x7f48b409dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:13.399 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.394+0000 7f48cdd9b700 1 --2- 172.21.15.120:0/28684971 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f48b409b700 0x7f48b409dbb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f48c4009b80 tx=0x7f48c400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:13.400 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.397+0000 7f48be7fc700 1 -- 172.21.15.120:0/28684971 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f48b8055a00 con 0x7f48c8102000 2023-06-23T01:18:13.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:13 smithi134 ceph-mon[116146]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:13.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:13 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:13.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:13 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:13.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:13 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:13.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:13 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T01:18:13.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:13 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T01:18:13.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:13 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:18:13.636 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.632+0000 7f48ced9d700 1 -- 172.21.15.120:0/28684971 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48c810a4f0 con 0x7f48c8102000 2023-06-23T01:18:13.636 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.632+0000 7f48be7fc700 1 -- 172.21.15.120:0/28684971 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4300 (secure 0 0 0) 0x7f48b8091030 con 0x7f48c8102000 2023-06-23T01:18:13.637 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:18:13.637 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14427,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1538161586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1538161586},{"type":"v1","addr":"172.21.15.120:6835","nonce":1538161586}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/2588400819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":2588400819},{"type":"v1","addr":"172.21.15.120:6837","nonce":2588400819}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3214085078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3214085078},{"type":"v1","addr":"172.21.15.134:6833","nonce":3214085078}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1478747127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1478747127},{"type":"v1","addr":"172.21.15.157:6833","nonce":1478747127}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/4238446318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":4238446318},{"type":"v1","addr":"172.21.15.134:6835","nonce":4238446318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2023-06-23T01:18:13.639 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.635+0000 7f48ced9d700 1 -- 172.21.15.120:0/28684971 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f48b409b700 msgr2=0x7f48b409dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:13.639 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.635+0000 7f48ced9d700 1 --2- 172.21.15.120:0/28684971 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f48b409b700 0x7f48b409dbb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f48c4009b80 tx=0x7f48c400b040 comp rx=0 tx=0).stop 2023-06-23T01:18:13.639 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.635+0000 7f48ced9d700 1 -- 172.21.15.120:0/28684971 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48c8102000 msgr2=0x7f48c810fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:13.639 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.635+0000 7f48ced9d700 1 --2- 172.21.15.120:0/28684971 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48c8102000 0x7f48c810fea0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f48b800b3b0 tx=0x7f48b8009e30 comp rx=0 tx=0).stop 2023-06-23T01:18:13.639 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.635+0000 7f48ced9d700 1 -- 172.21.15.120:0/28684971 shutdown_connections 2023-06-23T01:18:13.640 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.635+0000 7f48ced9d700 1 --2- 172.21.15.120:0/28684971 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f48b409b700 0x7f48b409dbb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:13.640 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.635+0000 7f48ced9d700 1 --2- 172.21.15.120:0/28684971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48c8104970 0x7f48c8116f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:13.640 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.635+0000 7f48ced9d700 1 --2- 172.21.15.120:0/28684971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48c8066bb0 0x7f48c810f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:13.640 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.635+0000 7f48ced9d700 1 --2- 172.21.15.120:0/28684971 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48c8102000 0x7f48c810fea0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:13.640 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.635+0000 7f48ced9d700 1 -- 172.21.15.120:0/28684971 >> 172.21.15.120:0/28684971 conn(0x7f48c80f7130 msgr2=0x7f48c8102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:13.640 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.635+0000 7f48ced9d700 1 -- 172.21.15.120:0/28684971 shutdown_connections 2023-06-23T01:18:13.640 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:13.635+0000 7f48ced9d700 1 -- 172.21.15.120:0/28684971 wait complete. 2023-06-23T01:18:13.641 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 6 2023-06-23T01:18:14.330 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:14 smithi157 ceph-mon[116945]: Deploying daemon mds.f on smithi157 2023-06-23T01:18:14.330 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:14 smithi157 ceph-mon[116945]: mds.? [v2:172.21.15.134:6834/4238446318,v1:172.21.15.134:6835/4238446318] up:boot 2023-06-23T01:18:14.331 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:14 smithi157 ceph-mon[116945]: fsmap 5 up:standby 2023-06-23T01:18:14.331 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:14 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-06-23T01:18:14.331 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:14 smithi157 ceph-mon[116945]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:14.331 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:14 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/28684971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:14.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:14 smithi134 ceph-mon[116146]: Deploying daemon mds.f on smithi157 2023-06-23T01:18:14.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:14 smithi134 ceph-mon[116146]: mds.? [v2:172.21.15.134:6834/4238446318,v1:172.21.15.134:6835/4238446318] up:boot 2023-06-23T01:18:14.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:14 smithi134 ceph-mon[116146]: fsmap 5 up:standby 2023-06-23T01:18:14.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:14 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-06-23T01:18:14.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:14 smithi134 ceph-mon[116146]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:14.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:14 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/28684971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:14.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:14 smithi120 ceph-mon[79813]: Deploying daemon mds.f on smithi157 2023-06-23T01:18:14.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:14 smithi120 ceph-mon[79813]: mds.? [v2:172.21.15.134:6834/4238446318,v1:172.21.15.134:6835/4238446318] up:boot 2023-06-23T01:18:14.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:14 smithi120 ceph-mon[79813]: fsmap 5 up:standby 2023-06-23T01:18:14.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:14 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-06-23T01:18:14.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:14 smithi120 ceph-mon[79813]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:14.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:14 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/28684971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:14.626 INFO:journalctl@ceph.mds.f.smithi157.stdout:Jun 23 01:18:14 smithi157 systemd[1]: Starting Ceph mds.f for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:18:15.045 INFO:journalctl@ceph.mds.f.smithi157.stdout:Jun 23 01:18:14 smithi157 bash[149858]: 3dec170d27311baaff691e0a93a0bfce389b022071cf35b545148f1fa6ecc47d 2023-06-23T01:18:15.046 INFO:journalctl@ceph.mds.f.smithi157.stdout:Jun 23 01:18:14 smithi157 ceph-mds[149923]: -- 172.21.15.157:0/1465369253 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5562cd1c09c0 con 0x5562cd196c00 2023-06-23T01:18:15.046 INFO:journalctl@ceph.mds.f.smithi157.stdout:Jun 23 01:18:14 smithi157 conmon[149898]: starting mds.f at 2023-06-23T01:18:15.376 INFO:journalctl@ceph.mds.f.smithi157.stdout:Jun 23 01:18:15 smithi157 systemd[1]: Started Ceph mds.f for 9efbfa8e-1162-11ee-9b2b-001a4aab830c. 2023-06-23T01:18:15.658 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T01:18:15.849 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.845+0000 7f48f3bff700 1 -- 172.21.15.120:0/2802382554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48ec1021f0 msgr2=0x7f48ec1045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:15.849 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.845+0000 7f48f3bff700 1 --2- 172.21.15.120:0/2802382554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48ec1021f0 0x7f48ec1045d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f48dc0049c0 tx=0x7f48dc009e30 comp rx=0 tx=0).stop 2023-06-23T01:18:15.849 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.845+0000 7f48f3bff700 1 -- 172.21.15.120:0/2802382554 shutdown_connections 2023-06-23T01:18:15.849 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.845+0000 7f48f3bff700 1 --2- 172.21.15.120:0/2802382554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48ec104b60 0x7f48ec106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:15.849 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.845+0000 7f48f3bff700 1 --2- 172.21.15.120:0/2802382554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48ec066e40 0x7f48ec101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:15.850 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.845+0000 7f48f3bff700 1 --2- 172.21.15.120:0/2802382554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48ec1021f0 0x7f48ec1045d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:15.850 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.845+0000 7f48f3bff700 1 -- 172.21.15.120:0/2802382554 >> 172.21.15.120:0/2802382554 conn(0x7f48ec0f7300 msgr2=0x7f48ec0f9750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:15.850 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.846+0000 7f48f3bff700 1 -- 172.21.15.120:0/2802382554 shutdown_connections 2023-06-23T01:18:15.850 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.846+0000 7f48f3bff700 1 -- 172.21.15.120:0/2802382554 wait complete. 2023-06-23T01:18:15.852 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.846+0000 7f48f3bff700 1 Processor -- start 2023-06-23T01:18:15.852 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.846+0000 7f48f3bff700 1 -- start start 2023-06-23T01:18:15.853 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.847+0000 7f48f3bff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48ec066e40 0x7f48ec10fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:15.853 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.847+0000 7f48f3bff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48ec1021f0 0x7f48ec1101e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:15.853 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.847+0000 7f48f3bff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48ec104b60 0x7f48ec117260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:15.853 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.847+0000 7f48f3bff700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec1178b0 con 0x7f48ec1021f0 2023-06-23T01:18:15.854 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.847+0000 7f48f3bff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec117a20 con 0x7f48ec066e40 2023-06-23T01:18:15.854 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.847+0000 7f48f3bff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec117be0 con 0x7f48ec104b60 2023-06-23T01:18:15.854 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.847+0000 7f48f119a700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48ec1021f0 0x7f48ec1101e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:15.855 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.847+0000 7f48f199b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48ec066e40 0x7f48ec10fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:15.855 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.847+0000 7f48f119a700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48ec1021f0 0x7f48ec1101e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38044/0 (socket says 172.21.15.120:38044) 2023-06-23T01:18:15.855 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.847+0000 7f48f119a700 1 -- 172.21.15.120:0/2303942610 learned_addr learned my addr 172.21.15.120:0/2303942610 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:18:15.855 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.847+0000 7f48f219c700 1 --2- 172.21.15.120:0/2303942610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48ec104b60 0x7f48ec117260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:15.856 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.847+0000 7f48f119a700 1 -- 172.21.15.120:0/2303942610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48ec104b60 msgr2=0x7f48ec117260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:15.856 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.847+0000 7f48f119a700 1 --2- 172.21.15.120:0/2303942610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48ec104b60 0x7f48ec117260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:15.856 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.847+0000 7f48f119a700 1 -- 172.21.15.120:0/2303942610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48ec066e40 msgr2=0x7f48ec10fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:15.857 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.847+0000 7f48f119a700 1 --2- 172.21.15.120:0/2303942610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48ec066e40 0x7f48ec10fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:15.857 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.847+0000 7f48f119a700 1 -- 172.21.15.120:0/2303942610 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48dc005040 con 0x7f48ec1021f0 2023-06-23T01:18:15.857 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.848+0000 7f48f119a700 1 --2- 172.21.15.120:0/2303942610 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48ec1021f0 0x7f48ec1101e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f48dc009430 tx=0x7f48dc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:15.857 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.848+0000 7f48e27fc700 1 -- 172.21.15.120:0/2303942610 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc020070 con 0x7f48ec1021f0 2023-06-23T01:18:15.857 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.848+0000 7f48f3bff700 1 -- 172.21.15.120:0/2303942610 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48ec117e60 con 0x7f48ec1021f0 2023-06-23T01:18:15.858 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.848+0000 7f48e27fc700 1 -- 172.21.15.120:0/2303942610 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48dc009700 con 0x7f48ec1021f0 2023-06-23T01:18:15.858 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.848+0000 7f48e27fc700 1 -- 172.21.15.120:0/2303942610 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc00e070 con 0x7f48ec1021f0 2023-06-23T01:18:15.858 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.848+0000 7f48f3bff700 1 -- 172.21.15.120:0/2303942610 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48ec118290 con 0x7f48ec1021f0 2023-06-23T01:18:15.858 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.849+0000 7f48f3bff700 1 -- 172.21.15.120:0/2303942610 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48ec0f8e30 con 0x7f48ec1021f0 2023-06-23T01:18:15.861 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.851+0000 7f48e27fc700 1 -- 172.21.15.120:0/2303942610 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7f48dc009870 con 0x7f48ec1021f0 2023-06-23T01:18:15.861 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.852+0000 7f48e27fc700 1 --2- 172.21.15.120:0/2303942610 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f48d809b680 0x7f48d809db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:15.861 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.852+0000 7f48e27fc700 1 -- 172.21.15.120:0/2303942610 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f48dc0c5a70 con 0x7f48ec1021f0 2023-06-23T01:18:15.861 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.852+0000 7f48f199b700 1 --2- 172.21.15.120:0/2303942610 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f48d809b680 0x7f48d809db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:15.861 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.853+0000 7f48f199b700 1 --2- 172.21.15.120:0/2303942610 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f48d809b680 0x7f48d809db30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f48ec110b90 tx=0x7f48e800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:15.861 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:15.856+0000 7f48e27fc700 1 -- 172.21.15.120:0/2303942610 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f48dc0916d0 con 0x7f48ec1021f0 2023-06-23T01:18:16.023 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:16 smithi157 ceph-mon[116945]: mds.? [v2:172.21.15.157:6834/2129674435,v1:172.21.15.157:6835/2129674435] up:boot 2023-06-23T01:18:16.023 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:16 smithi157 ceph-mon[116945]: fsmap 6 up:standby 2023-06-23T01:18:16.023 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:16 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-06-23T01:18:16.024 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:16 smithi157 ceph-mon[116945]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:16.024 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:16 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:16.024 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:16 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:16.024 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:16 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:16.024 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:16 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T01:18:16.024 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:16 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T01:18:16.024 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:16 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:18:16.025 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:16 smithi157 ceph-mon[116945]: Deploying daemon mds.c on smithi157 2023-06-23T01:18:16.098 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:16.095+0000 7f48f3bff700 1 -- 172.21.15.120:0/2303942610 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48ec10a730 con 0x7f48ec1021f0 2023-06-23T01:18:16.098 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:16.095+0000 7f48e27fc700 1 -- 172.21.15.120:0/2303942610 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5060 (secure 0 0 0) 0x7f48dc094cf0 con 0x7f48ec1021f0 2023-06-23T01:18:16.099 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:18:16.099 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14427,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1538161586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1538161586},{"type":"v1","addr":"172.21.15.120:6835","nonce":1538161586}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/2588400819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":2588400819},{"type":"v1","addr":"172.21.15.120:6837","nonce":2588400819}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3214085078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3214085078},{"type":"v1","addr":"172.21.15.134:6833","nonce":3214085078}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1478747127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1478747127},{"type":"v1","addr":"172.21.15.157:6833","nonce":1478747127}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/4238446318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":4238446318},{"type":"v1","addr":"172.21.15.134:6835","nonce":4238446318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2129674435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2129674435},{"type":"v1","addr":"172.21.15.157:6835","nonce":2129674435}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[]} 2023-06-23T01:18:16.101 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:16.098+0000 7f48f3bff700 1 -- 172.21.15.120:0/2303942610 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f48d809b680 msgr2=0x7f48d809db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:16.101 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:16.098+0000 7f48f3bff700 1 --2- 172.21.15.120:0/2303942610 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f48d809b680 0x7f48d809db30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f48ec110b90 tx=0x7f48e800b040 comp rx=0 tx=0).stop 2023-06-23T01:18:16.101 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:16.098+0000 7f48f3bff700 1 -- 172.21.15.120:0/2303942610 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48ec1021f0 msgr2=0x7f48ec1101e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:16.101 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:16.098+0000 7f48f3bff700 1 --2- 172.21.15.120:0/2303942610 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48ec1021f0 0x7f48ec1101e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f48dc009430 tx=0x7f48dc009e30 comp rx=0 tx=0).stop 2023-06-23T01:18:16.101 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:16.098+0000 7f48f3bff700 1 -- 172.21.15.120:0/2303942610 shutdown_connections 2023-06-23T01:18:16.101 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:16.098+0000 7f48f3bff700 1 --2- 172.21.15.120:0/2303942610 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f48d809b680 0x7f48d809db30 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:16.101 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:16.098+0000 7f48f3bff700 1 --2- 172.21.15.120:0/2303942610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48ec104b60 0x7f48ec117260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:16.102 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:16.098+0000 7f48f3bff700 1 --2- 172.21.15.120:0/2303942610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48ec066e40 0x7f48ec10fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:16.102 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:16.098+0000 7f48f3bff700 1 --2- 172.21.15.120:0/2303942610 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48ec1021f0 0x7f48ec1101e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:16.102 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:16.098+0000 7f48f3bff700 1 -- 172.21.15.120:0/2303942610 >> 172.21.15.120:0/2303942610 conn(0x7f48ec0f7300 msgr2=0x7f48ec105900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:16.102 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:16.098+0000 7f48f3bff700 1 -- 172.21.15.120:0/2303942610 shutdown_connections 2023-06-23T01:18:16.102 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:16.098+0000 7f48f3bff700 1 -- 172.21.15.120:0/2303942610 wait complete. 2023-06-23T01:18:16.103 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 7 2023-06-23T01:18:16.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:16 smithi134 ceph-mon[116146]: mds.? [v2:172.21.15.157:6834/2129674435,v1:172.21.15.157:6835/2129674435] up:boot 2023-06-23T01:18:16.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:16 smithi134 ceph-mon[116146]: fsmap 6 up:standby 2023-06-23T01:18:16.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:16 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-06-23T01:18:16.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:16 smithi134 ceph-mon[116146]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:16.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:16 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:16.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:16 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:16.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:16 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:16.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:16 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T01:18:16.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:16 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T01:18:16.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:16 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:18:16.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:16 smithi134 ceph-mon[116146]: Deploying daemon mds.c on smithi157 2023-06-23T01:18:16.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:16 smithi120 ceph-mon[79813]: mds.? [v2:172.21.15.157:6834/2129674435,v1:172.21.15.157:6835/2129674435] up:boot 2023-06-23T01:18:16.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:16 smithi120 ceph-mon[79813]: fsmap 6 up:standby 2023-06-23T01:18:16.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:16 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-06-23T01:18:16.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:16 smithi120 ceph-mon[79813]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:16.512 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:16 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:16.512 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:16 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:16.512 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:16 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:16.513 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:16 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T01:18:16.513 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:16 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T01:18:16.513 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:16 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:18:16.513 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:16 smithi120 ceph-mon[79813]: Deploying daemon mds.c on smithi157 2023-06-23T01:18:17.318 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:17 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/2303942610' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:17.318 INFO:journalctl@ceph.mds.c.smithi157.stdout:Jun 23 01:18:16 smithi157 systemd[1]: Starting Ceph mds.c for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:18:17.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:17 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/2303942610' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:17.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:17 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/2303942610' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:17.626 INFO:journalctl@ceph.mds.c.smithi157.stdout:Jun 23 01:18:17 smithi157 ceph-mds[150485]: -- 172.21.15.157:0/182859529 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561c337ac9c0 con 0x561c33782800 2023-06-23T01:18:17.627 INFO:journalctl@ceph.mds.c.smithi157.stdout:Jun 23 01:18:17 smithi157 conmon[150461]: starting mds.c at 2023-06-23T01:18:17.627 INFO:journalctl@ceph.mds.c.smithi157.stdout:Jun 23 01:18:17 smithi157 bash[150421]: 92040a2e4e7855f73c307ea289f6cd00417696e6c7e047a0d6702405d5ec64ee 2023-06-23T01:18:17.627 INFO:journalctl@ceph.mds.c.smithi157.stdout:Jun 23 01:18:17 smithi157 systemd[1]: Started Ceph mds.c for 9efbfa8e-1162-11ee-9b2b-001a4aab830c. 2023-06-23T01:18:18.119 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T01:18:18.313 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.309+0000 7ff2331ff700 1 -- 172.21.15.120:0/3910477467 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff22c0fcca0 msgr2=0x7ff22c0fd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:18.313 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.309+0000 7ff2331ff700 1 --2- 172.21.15.120:0/3910477467 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff22c0fcca0 0x7ff22c0fd0b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff2200049c0 tx=0x7ff220009e30 comp rx=0 tx=0).stop 2023-06-23T01:18:18.313 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.309+0000 7ff2331ff700 1 -- 172.21.15.120:0/3910477467 shutdown_connections 2023-06-23T01:18:18.313 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.309+0000 7ff2331ff700 1 --2- 172.21.15.120:0/3910477467 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff22c0fcca0 0x7ff22c0fd0b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:18.314 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.309+0000 7ff2331ff700 1 --2- 172.21.15.120:0/3910477467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff22c0fe710 0x7ff22c105230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:18.314 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.309+0000 7ff2331ff700 1 --2- 172.21.15.120:0/3910477467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff22c0fd5f0 0x7ff22c0fdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:18.314 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.309+0000 7ff2331ff700 1 -- 172.21.15.120:0/3910477467 >> 172.21.15.120:0/3910477467 conn(0x7ff22c0f8470 msgr2=0x7ff22c0fa8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:18.314 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.310+0000 7ff2331ff700 1 -- 172.21.15.120:0/3910477467 shutdown_connections 2023-06-23T01:18:18.314 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.310+0000 7ff2331ff700 1 -- 172.21.15.120:0/3910477467 wait complete. 2023-06-23T01:18:18.314 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.310+0000 7ff2331ff700 1 Processor -- start 2023-06-23T01:18:18.316 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.310+0000 7ff2331ff700 1 -- start start 2023-06-23T01:18:18.316 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.311+0000 7ff2331ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff22c0fcca0 0x7ff22c10d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:18.316 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.311+0000 7ff2331ff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff22c0fd5f0 0x7ff22c10de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:18.316 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.311+0000 7ff2331ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff22c0fe710 0x7ff22c114ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:18.316 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.311+0000 7ff2331ff700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22c1154f0 con 0x7ff22c0fd5f0 2023-06-23T01:18:18.317 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.311+0000 7ff2331ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22c115660 con 0x7ff22c0fcca0 2023-06-23T01:18:18.317 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.311+0000 7ff2331ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22c115820 con 0x7ff22c0fe710 2023-06-23T01:18:18.317 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.311+0000 7ff230f9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff22c0fcca0 0x7ff22c10d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:18.317 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.311+0000 7ff22bfff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff22c0fd5f0 0x7ff22c10de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:18.317 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.311+0000 7ff230f9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff22c0fcca0 0x7ff22c10d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:60652/0 (socket says 172.21.15.120:60652) 2023-06-23T01:18:18.317 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.311+0000 7ff230f9b700 1 -- 172.21.15.120:0/738970249 learned_addr learned my addr 172.21.15.120:0/738970249 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:18:18.317 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.311+0000 7ff23179c700 1 --2- 172.21.15.120:0/738970249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff22c0fe710 0x7ff22c114ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:18.317 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.311+0000 7ff230f9b700 1 -- 172.21.15.120:0/738970249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff22c0fe710 msgr2=0x7ff22c114ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:18.318 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.311+0000 7ff230f9b700 1 --2- 172.21.15.120:0/738970249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff22c0fe710 0x7ff22c114ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:18.318 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.311+0000 7ff230f9b700 1 -- 172.21.15.120:0/738970249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff22c0fd5f0 msgr2=0x7ff22c10de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:18.318 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.311+0000 7ff230f9b700 1 --2- 172.21.15.120:0/738970249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff22c0fd5f0 0x7ff22c10de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:18.318 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.311+0000 7ff230f9b700 1 -- 172.21.15.120:0/738970249 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff220005040 con 0x7ff22c0fcca0 2023-06-23T01:18:18.318 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.311+0000 7ff22bfff700 1 --2- 172.21.15.120:0/738970249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff22c0fd5f0 0x7ff22c10de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T01:18:18.318 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.311+0000 7ff230f9b700 1 --2- 172.21.15.120:0/738970249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff22c0fcca0 0x7ff22c10d910 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff22000b920 tx=0x7ff220009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:18.318 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.312+0000 7ff2297fa700 1 -- 172.21.15.120:0/738970249 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22000e040 con 0x7ff22c0fcca0 2023-06-23T01:18:18.319 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.312+0000 7ff2331ff700 1 -- 172.21.15.120:0/738970249 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff22c115aa0 con 0x7ff22c0fcca0 2023-06-23T01:18:18.319 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.312+0000 7ff2297fa700 1 -- 172.21.15.120:0/738970249 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff220013420 con 0x7ff22c0fcca0 2023-06-23T01:18:18.319 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.312+0000 7ff2297fa700 1 -- 172.21.15.120:0/738970249 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2200096f0 con 0x7ff22c0fcca0 2023-06-23T01:18:18.319 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.312+0000 7ff2331ff700 1 -- 172.21.15.120:0/738970249 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff22c115f80 con 0x7ff22c0fcca0 2023-06-23T01:18:18.319 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.313+0000 7ff2331ff700 1 -- 172.21.15.120:0/738970249 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff22c1042d0 con 0x7ff22c0fcca0 2023-06-23T01:18:18.320 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.314+0000 7ff2297fa700 1 -- 172.21.15.120:0/738970249 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7ff220013800 con 0x7ff22c0fcca0 2023-06-23T01:18:18.320 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.316+0000 7ff2297fa700 1 --2- 172.21.15.120:0/738970249 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7ff21c09b680 0x7ff21c09db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:18.320 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.316+0000 7ff2297fa700 1 -- 172.21.15.120:0/738970249 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7ff2200c5c50 con 0x7ff22c0fcca0 2023-06-23T01:18:18.320 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.316+0000 7ff22bfff700 1 --2- 172.21.15.120:0/738970249 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7ff21c09b680 0x7ff21c09db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:18.320 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.316+0000 7ff22bfff700 1 --2- 172.21.15.120:0/738970249 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7ff21c09b680 0x7ff21c09db30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff22c0fd9e0 tx=0x7ff21800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:18.321 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.318+0000 7ff2297fa700 1 -- 172.21.15.120:0/738970249 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7ff2200918b0 con 0x7ff22c0fcca0 2023-06-23T01:18:18.556 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.554+0000 7ff2331ff700 1 -- 172.21.15.120:0/738970249 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff22c108500 con 0x7ff22c0fcca0 2023-06-23T01:18:18.557 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.555+0000 7ff2297fa700 1 -- 172.21.15.120:0/738970249 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5820 (secure 0 0 0) 0x7ff220094ed0 con 0x7ff22c0fcca0 2023-06-23T01:18:18.558 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:18:18.558 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14427,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1538161586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1538161586},{"type":"v1","addr":"172.21.15.120:6835","nonce":1538161586}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/2588400819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":2588400819},{"type":"v1","addr":"172.21.15.120:6837","nonce":2588400819}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3214085078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3214085078},{"type":"v1","addr":"172.21.15.134:6833","nonce":3214085078}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1478747127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1478747127},{"type":"v1","addr":"172.21.15.157:6833","nonce":1478747127}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/4238446318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":4238446318},{"type":"v1","addr":"172.21.15.134:6835","nonce":4238446318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2129674435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2129674435},{"type":"v1","addr":"172.21.15.157:6835","nonce":2129674435}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1606323168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1606323168},{"type":"v1","addr":"172.21.15.157:6837","nonce":1606323168}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2023-06-23T01:18:18.560 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.557+0000 7ff2331ff700 1 -- 172.21.15.120:0/738970249 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7ff21c09b680 msgr2=0x7ff21c09db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:18.560 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.557+0000 7ff2331ff700 1 --2- 172.21.15.120:0/738970249 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7ff21c09b680 0x7ff21c09db30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff22c0fd9e0 tx=0x7ff21800b040 comp rx=0 tx=0).stop 2023-06-23T01:18:18.560 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.557+0000 7ff2331ff700 1 -- 172.21.15.120:0/738970249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff22c0fcca0 msgr2=0x7ff22c10d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:18.560 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.557+0000 7ff2331ff700 1 --2- 172.21.15.120:0/738970249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff22c0fcca0 0x7ff22c10d910 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff22000b920 tx=0x7ff220009e30 comp rx=0 tx=0).stop 2023-06-23T01:18:18.560 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.557+0000 7ff2331ff700 1 -- 172.21.15.120:0/738970249 shutdown_connections 2023-06-23T01:18:18.561 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.557+0000 7ff2331ff700 1 --2- 172.21.15.120:0/738970249 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7ff21c09b680 0x7ff21c09db30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:18.561 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.557+0000 7ff2331ff700 1 --2- 172.21.15.120:0/738970249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff22c0fe710 0x7ff22c114ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:18.561 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.557+0000 7ff2331ff700 1 --2- 172.21.15.120:0/738970249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff22c0fcca0 0x7ff22c10d910 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:18.561 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.557+0000 7ff2331ff700 1 --2- 172.21.15.120:0/738970249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff22c0fd5f0 0x7ff22c10de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:18.561 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.557+0000 7ff2331ff700 1 -- 172.21.15.120:0/738970249 >> 172.21.15.120:0/738970249 conn(0x7ff22c0f8470 msgr2=0x7ff22c101450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:18.561 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.557+0000 7ff2331ff700 1 -- 172.21.15.120:0/738970249 shutdown_connections 2023-06-23T01:18:18.562 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:18.557+0000 7ff2331ff700 1 -- 172.21.15.120:0/738970249 wait complete. 2023-06-23T01:18:18.562 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 8 2023-06-23T01:18:18.876 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:18 smithi157 ceph-mon[116945]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:18.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:18 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:18.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:18 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:18.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:18 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:18.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:18 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T01:18:18.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:18 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T01:18:18.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:18 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:18:18.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:18 smithi157 ceph-mon[116945]: Deploying daemon mds.h on smithi134 2023-06-23T01:18:18.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:18 smithi157 ceph-mon[116945]: mds.? [v2:172.21.15.157:6836/1606323168,v1:172.21.15.157:6837/1606323168] up:boot 2023-06-23T01:18:18.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:18 smithi157 ceph-mon[116945]: fsmap 7 up:standby 2023-06-23T01:18:18.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:18 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-06-23T01:18:18.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:18 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/738970249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:18.943 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:18 smithi134 ceph-mon[116146]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:18.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:18 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:18.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:18 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:18.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:18 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:18.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:18 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T01:18:18.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:18 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T01:18:18.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:18 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:18:18.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:18 smithi134 ceph-mon[116146]: Deploying daemon mds.h on smithi134 2023-06-23T01:18:18.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:18 smithi134 ceph-mon[116146]: mds.? [v2:172.21.15.157:6836/1606323168,v1:172.21.15.157:6837/1606323168] up:boot 2023-06-23T01:18:18.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:18 smithi134 ceph-mon[116146]: fsmap 7 up:standby 2023-06-23T01:18:18.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:18 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-06-23T01:18:18.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:18 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/738970249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:19.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:18 smithi120 ceph-mon[79813]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:19.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:18 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:19.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:18 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:19.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:18 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:19.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:18 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T01:18:19.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:18 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T01:18:19.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:18 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:18:19.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:18 smithi120 ceph-mon[79813]: Deploying daemon mds.h on smithi134 2023-06-23T01:18:19.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:18 smithi120 ceph-mon[79813]: mds.? [v2:172.21.15.157:6836/1606323168,v1:172.21.15.157:6837/1606323168] up:boot 2023-06-23T01:18:19.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:18 smithi120 ceph-mon[79813]: fsmap 7 up:standby 2023-06-23T01:18:19.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:18 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-06-23T01:18:19.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:18 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/738970249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:19.496 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jun 23 01:18:19 smithi134 systemd[1]: Starting Ceph mds.h for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:18:20.193 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jun 23 01:18:20 smithi134 ceph-mds[148389]: -- 172.21.15.134:0/2736328070 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55568ef1c9c0 con 0x55568eef3000 2023-06-23T01:18:20.194 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jun 23 01:18:20 smithi134 conmon[148364]: starting mds.h at 2023-06-23T01:18:20.194 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jun 23 01:18:20 smithi134 bash[148323]: 94b1beee6fb79ba49df8698b1dd71af9b65e287a869438c16d59dd932512e318 2023-06-23T01:18:20.194 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jun 23 01:18:20 smithi134 systemd[1]: Started Ceph mds.h for 9efbfa8e-1162-11ee-9b2b-001a4aab830c. 2023-06-23T01:18:20.579 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T01:18:20.773 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.770+0000 7fc826dff700 1 -- 172.21.15.120:0/1287722299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8200fe970 msgr2=0x7fc820105210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:20.773 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.770+0000 7fc826dff700 1 --2- 172.21.15.120:0/1287722299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8200fe970 0x7fc820105210 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc818009d60 tx=0x7fc818004d10 comp rx=0 tx=0).stop 2023-06-23T01:18:20.774 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.770+0000 7fc826dff700 1 -- 172.21.15.120:0/1287722299 shutdown_connections 2023-06-23T01:18:20.774 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.770+0000 7fc826dff700 1 --2- 172.21.15.120:0/1287722299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8200fe970 0x7fc820105210 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:20.774 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.770+0000 7fc826dff700 1 --2- 172.21.15.120:0/1287722299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8200fcd10 0x7fc8200fd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:20.774 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.770+0000 7fc826dff700 1 --2- 172.21.15.120:0/1287722299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8200fdfc0 0x7fc8200fe430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:20.774 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.770+0000 7fc826dff700 1 -- 172.21.15.120:0/1287722299 >> 172.21.15.120:0/1287722299 conn(0x7fc8200f8490 msgr2=0x7fc8200fa8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:20.774 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.770+0000 7fc826dff700 1 -- 172.21.15.120:0/1287722299 shutdown_connections 2023-06-23T01:18:20.774 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.770+0000 7fc826dff700 1 -- 172.21.15.120:0/1287722299 wait complete. 2023-06-23T01:18:20.776 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.771+0000 7fc826dff700 1 Processor -- start 2023-06-23T01:18:20.776 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.771+0000 7fc826dff700 1 -- start start 2023-06-23T01:18:20.776 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.771+0000 7fc826dff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8200fcd10 0x7fc82010f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:20.777 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.771+0000 7fc826dff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8200fdfc0 0x7fc82010fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:20.777 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.771+0000 7fc826dff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8200fe970 0x7fc820116f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:20.777 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.771+0000 7fc826dff700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc820117570 con 0x7fc8200fe970 2023-06-23T01:18:20.777 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.771+0000 7fc826dff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8201176e0 con 0x7fc8200fcd10 2023-06-23T01:18:20.777 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.771+0000 7fc826dff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8201178a0 con 0x7fc8200fdfc0 2023-06-23T01:18:20.777 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.772+0000 7fc81ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8200fdfc0 0x7fc82010fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:20.777 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.772+0000 7fc81ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8200fdfc0 0x7fc82010fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.120:39238/0 (socket says 172.21.15.120:39238) 2023-06-23T01:18:20.777 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.772+0000 7fc81ffff700 1 -- 172.21.15.120:0/2790749672 learned_addr learned my addr 172.21.15.120:0/2790749672 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:18:20.778 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.772+0000 7fc824b9b700 1 --2- 172.21.15.120:0/2790749672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8200fcd10 0x7fc82010f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:20.778 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.772+0000 7fc82539c700 1 --2- 172.21.15.120:0/2790749672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8200fe970 0x7fc820116f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:20.778 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.772+0000 7fc81ffff700 1 -- 172.21.15.120:0/2790749672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8200fcd10 msgr2=0x7fc82010f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:20.778 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.772+0000 7fc81ffff700 1 --2- 172.21.15.120:0/2790749672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8200fcd10 0x7fc82010f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:20.778 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.772+0000 7fc81ffff700 1 -- 172.21.15.120:0/2790749672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8200fe970 msgr2=0x7fc820116f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:20.778 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.772+0000 7fc81ffff700 1 --2- 172.21.15.120:0/2790749672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8200fe970 0x7fc820116f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:20.778 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.772+0000 7fc81ffff700 1 -- 172.21.15.120:0/2790749672 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc818009940 con 0x7fc8200fdfc0 2023-06-23T01:18:20.779 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.772+0000 7fc824b9b700 1 --2- 172.21.15.120:0/2790749672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8200fcd10 0x7fc82010f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T01:18:20.779 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.772+0000 7fc81ffff700 1 --2- 172.21.15.120:0/2790749672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8200fdfc0 0x7fc82010fed0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc80c00ebc0 tx=0x7fc80c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:20.779 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.772+0000 7fc81d7fa700 1 -- 172.21.15.120:0/2790749672 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c007cc0 con 0x7fc8200fdfc0 2023-06-23T01:18:20.779 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.772+0000 7fc826dff700 1 -- 172.21.15.120:0/2790749672 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc820117b20 con 0x7fc8200fdfc0 2023-06-23T01:18:20.779 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.772+0000 7fc81d7fa700 1 -- 172.21.15.120:0/2790749672 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc80c00f040 con 0x7fc8200fdfc0 2023-06-23T01:18:20.779 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.772+0000 7fc81d7fa700 1 -- 172.21.15.120:0/2790749672 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c0094a0 con 0x7fc8200fdfc0 2023-06-23T01:18:20.779 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.773+0000 7fc826dff700 1 -- 172.21.15.120:0/2790749672 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc820118060 con 0x7fc8200fdfc0 2023-06-23T01:18:20.779 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.773+0000 7fc826dff700 1 -- 172.21.15.120:0/2790749672 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc820108570 con 0x7fc8200fdfc0 2023-06-23T01:18:20.780 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.776+0000 7fc81d7fa700 1 -- 172.21.15.120:0/2790749672 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7fc80c00c600 con 0x7fc8200fdfc0 2023-06-23T01:18:20.780 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.777+0000 7fc81d7fa700 1 --2- 172.21.15.120:0/2790749672 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fc81009b700 0x7fc81009dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:20.781 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.777+0000 7fc81d7fa700 1 -- 172.21.15.120:0/2790749672 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7fc80c0c4660 con 0x7fc8200fdfc0 2023-06-23T01:18:20.781 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.777+0000 7fc824b9b700 1 --2- 172.21.15.120:0/2790749672 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fc81009b700 0x7fc81009dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:20.781 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.778+0000 7fc824b9b700 1 --2- 172.21.15.120:0/2790749672 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fc81009b700 0x7fc81009dbb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc814004600 tx=0x7fc81400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:20.781 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:20.779+0000 7fc81d7fa700 1 -- 172.21.15.120:0/2790749672 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fc80c04da00 con 0x7fc8200fdfc0 2023-06-23T01:18:20.876 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:20 smithi157 ceph-mon[116945]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:20.876 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:20 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:20.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:20 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:20.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:20 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:20.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:20 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T01:18:20.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:20 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T01:18:20.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:20 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:18:20.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:20 smithi134 ceph-mon[116146]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:20.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:20 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:20.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:20 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:20.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:20 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:20.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:20 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T01:18:20.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:20 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T01:18:20.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:20 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:18:21.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:20 smithi120 ceph-mon[79813]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:21.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:20 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:21.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:20 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:21.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:20 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:21.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:20 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T01:18:21.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:20 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T01:18:21.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:20 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:18:21.021 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:21.018+0000 7fc826dff700 1 -- 172.21.15.120:0/2790749672 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc820118250 con 0x7fc8200fdfc0 2023-06-23T01:18:21.022 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:21.019+0000 7fc81d7fa700 1 -- 172.21.15.120:0/2790749672 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6580 (secure 0 0 0) 0x7fc80c019090 con 0x7fc8200fdfc0 2023-06-23T01:18:21.022 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:18:21.023 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14427,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1538161586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1538161586},{"type":"v1","addr":"172.21.15.120:6835","nonce":1538161586}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/2588400819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":2588400819},{"type":"v1","addr":"172.21.15.120:6837","nonce":2588400819}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3214085078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3214085078},{"type":"v1","addr":"172.21.15.134:6833","nonce":3214085078}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1478747127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1478747127},{"type":"v1","addr":"172.21.15.157:6833","nonce":1478747127}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/4238446318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":4238446318},{"type":"v1","addr":"172.21.15.134:6835","nonce":4238446318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2129674435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2129674435},{"type":"v1","addr":"172.21.15.157:6835","nonce":2129674435}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3499504142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3499504142},{"type":"v1","addr":"172.21.15.134:6837","nonce":3499504142}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1606323168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1606323168},{"type":"v1","addr":"172.21.15.157:6837","nonce":1606323168}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2023-06-23T01:18:21.025 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:21.022+0000 7fc826dff700 1 -- 172.21.15.120:0/2790749672 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fc81009b700 msgr2=0x7fc81009dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:21.025 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:21.022+0000 7fc826dff700 1 --2- 172.21.15.120:0/2790749672 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fc81009b700 0x7fc81009dbb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc814004600 tx=0x7fc81400b040 comp rx=0 tx=0).stop 2023-06-23T01:18:21.025 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:21.022+0000 7fc826dff700 1 -- 172.21.15.120:0/2790749672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8200fdfc0 msgr2=0x7fc82010fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:21.025 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:21.022+0000 7fc826dff700 1 --2- 172.21.15.120:0/2790749672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8200fdfc0 0x7fc82010fed0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc80c00ebc0 tx=0x7fc80c00b040 comp rx=0 tx=0).stop 2023-06-23T01:18:21.025 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:21.022+0000 7fc826dff700 1 -- 172.21.15.120:0/2790749672 shutdown_connections 2023-06-23T01:18:21.026 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:21.022+0000 7fc826dff700 1 --2- 172.21.15.120:0/2790749672 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fc81009b700 0x7fc81009dbb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:21.026 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:21.022+0000 7fc826dff700 1 --2- 172.21.15.120:0/2790749672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8200fe970 0x7fc820116f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:21.026 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:21.022+0000 7fc826dff700 1 --2- 172.21.15.120:0/2790749672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8200fcd10 0x7fc82010f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:21.026 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:21.022+0000 7fc826dff700 1 --2- 172.21.15.120:0/2790749672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8200fdfc0 0x7fc82010fed0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:21.026 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:21.022+0000 7fc826dff700 1 -- 172.21.15.120:0/2790749672 >> 172.21.15.120:0/2790749672 conn(0x7fc8200f8490 msgr2=0x7fc82006fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:21.026 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:21.022+0000 7fc826dff700 1 -- 172.21.15.120:0/2790749672 shutdown_connections 2023-06-23T01:18:21.026 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:21.022+0000 7fc826dff700 1 -- 172.21.15.120:0/2790749672 wait complete. 2023-06-23T01:18:21.027 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 9 2023-06-23T01:18:21.932 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:21 smithi157 ceph-mon[116945]: Deploying daemon mds.i on smithi157 2023-06-23T01:18:21.932 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:21 smithi157 ceph-mon[116945]: mds.? [v2:172.21.15.134:6836/3499504142,v1:172.21.15.134:6837/3499504142] up:boot 2023-06-23T01:18:21.933 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:21 smithi157 ceph-mon[116945]: fsmap 8 up:standby 2023-06-23T01:18:21.933 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:21 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-06-23T01:18:21.933 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:21 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/2790749672' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:21.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:21 smithi134 ceph-mon[116146]: Deploying daemon mds.i on smithi157 2023-06-23T01:18:21.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:21 smithi134 ceph-mon[116146]: mds.? [v2:172.21.15.134:6836/3499504142,v1:172.21.15.134:6837/3499504142] up:boot 2023-06-23T01:18:21.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:21 smithi134 ceph-mon[116146]: fsmap 8 up:standby 2023-06-23T01:18:21.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:21 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-06-23T01:18:21.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:21 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/2790749672' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:22.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:21 smithi120 ceph-mon[79813]: Deploying daemon mds.i on smithi157 2023-06-23T01:18:22.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:21 smithi120 ceph-mon[79813]: mds.? [v2:172.21.15.134:6836/3499504142,v1:172.21.15.134:6837/3499504142] up:boot 2023-06-23T01:18:22.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:21 smithi120 ceph-mon[79813]: fsmap 8 up:standby 2023-06-23T01:18:22.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:21 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-06-23T01:18:22.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:21 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/2790749672' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:22.377 INFO:journalctl@ceph.mds.i.smithi157.stdout:Jun 23 01:18:22 smithi157 systemd[1]: Starting Ceph mds.i for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:18:22.659 INFO:journalctl@ceph.mds.i.smithi157.stdout:Jun 23 01:18:22 smithi157 ceph-mds[151046]: -- 172.21.15.157:0/4123071741 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5580c61809c0 con 0x5580c6157000 2023-06-23T01:18:22.660 INFO:journalctl@ceph.mds.i.smithi157.stdout:Jun 23 01:18:22 smithi157 conmon[151022]: starting mds.i at 2023-06-23T01:18:22.660 INFO:journalctl@ceph.mds.i.smithi157.stdout:Jun 23 01:18:22 smithi157 bash[150982]: 1ce0b23d18280193e6d9b739ca49d6efef908b79dd7d267114fb61767f3af4ed 2023-06-23T01:18:22.993 INFO:journalctl@ceph.mds.i.smithi157.stdout:Jun 23 01:18:22 smithi157 systemd[1]: Started Ceph mds.i for 9efbfa8e-1162-11ee-9b2b-001a4aab830c. 2023-06-23T01:18:23.045 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T01:18:23.236 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.233+0000 7fa2349ff700 1 -- 172.21.15.120:0/367490649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa230104ce0 msgr2=0x7fa2301070c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:23.237 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.233+0000 7fa2349ff700 1 --2- 172.21.15.120:0/367490649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa230104ce0 0x7fa2301070c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa228009d60 tx=0x7fa228004d10 comp rx=0 tx=0).stop 2023-06-23T01:18:23.237 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.233+0000 7fa2349ff700 1 -- 172.21.15.120:0/367490649 shutdown_connections 2023-06-23T01:18:23.237 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.233+0000 7fa2349ff700 1 --2- 172.21.15.120:0/367490649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa230104ce0 0x7fa2301070c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:23.237 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.233+0000 7fa2349ff700 1 --2- 172.21.15.120:0/367490649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa230071820 0x7fa23006fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:23.237 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.233+0000 7fa2349ff700 1 --2- 172.21.15.120:0/367490649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa230070300 0x7fa230070770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:23.238 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.233+0000 7fa2349ff700 1 -- 172.21.15.120:0/367490649 >> 172.21.15.120:0/367490649 conn(0x7fa2300f84a0 msgr2=0x7fa2300fa8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:23.238 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.233+0000 7fa2349ff700 1 -- 172.21.15.120:0/367490649 shutdown_connections 2023-06-23T01:18:23.238 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.233+0000 7fa2349ff700 1 -- 172.21.15.120:0/367490649 wait complete. 2023-06-23T01:18:23.239 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.235+0000 7fa2349ff700 1 Processor -- start 2023-06-23T01:18:23.240 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.235+0000 7fa2349ff700 1 -- start start 2023-06-23T01:18:23.240 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.235+0000 7fa2349ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa230070300 0x7fa23010f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:23.240 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.235+0000 7fa2349ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa230071820 0x7fa23010fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:23.240 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.235+0000 7fa2349ff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa230104ce0 0x7fa230116f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:23.240 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.235+0000 7fa2349ff700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa230117580 con 0x7fa230104ce0 2023-06-23T01:18:23.240 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.235+0000 7fa2349ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2301176f0 con 0x7fa230071820 2023-06-23T01:18:23.241 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.235+0000 7fa2349ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2301178b0 con 0x7fa230070300 2023-06-23T01:18:23.241 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.235+0000 7fa22ed9d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa230104ce0 0x7fa230116f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:23.241 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.235+0000 7fa22e59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa230070300 0x7fa23010f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:23.241 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.235+0000 7fa22e59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa230070300 0x7fa23010f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.120:39490/0 (socket says 172.21.15.120:39490) 2023-06-23T01:18:23.241 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.235+0000 7fa22e59c700 1 -- 172.21.15.120:0/3142370421 learned_addr learned my addr 172.21.15.120:0/3142370421 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:18:23.241 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.235+0000 7fa22dd9b700 1 --2- 172.21.15.120:0/3142370421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa230071820 0x7fa23010fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:23.241 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.236+0000 7fa22e59c700 1 -- 172.21.15.120:0/3142370421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa230071820 msgr2=0x7fa23010fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:23.242 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.236+0000 7fa22e59c700 1 --2- 172.21.15.120:0/3142370421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa230071820 0x7fa23010fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:23.242 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.236+0000 7fa22e59c700 1 -- 172.21.15.120:0/3142370421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa230104ce0 msgr2=0x7fa230116f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:23.242 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.236+0000 7fa22e59c700 1 --2- 172.21.15.120:0/3142370421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa230104ce0 0x7fa230116f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:23.242 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.236+0000 7fa22e59c700 1 -- 172.21.15.120:0/3142370421 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa224005040 con 0x7fa230070300 2023-06-23T01:18:23.242 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.236+0000 7fa22ed9d700 1 --2- 172.21.15.120:0/3142370421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa230104ce0 0x7fa230116f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T01:18:23.242 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.236+0000 7fa22e59c700 1 --2- 172.21.15.120:0/3142370421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa230070300 0x7fa23010f9a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa22000ebf0 tx=0x7fa22000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:23.242 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.236+0000 7fa21effd700 1 -- 172.21.15.120:0/3142370421 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2200047d0 con 0x7fa230070300 2023-06-23T01:18:23.243 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.236+0000 7fa2349ff700 1 -- 172.21.15.120:0/3142370421 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa228009940 con 0x7fa230070300 2023-06-23T01:18:23.243 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.236+0000 7fa21effd700 1 -- 172.21.15.120:0/3142370421 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa22000f040 con 0x7fa230070300 2023-06-23T01:18:23.243 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.236+0000 7fa21effd700 1 -- 172.21.15.120:0/3142370421 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22000c980 con 0x7fa230070300 2023-06-23T01:18:23.243 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.236+0000 7fa2349ff700 1 -- 172.21.15.120:0/3142370421 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa230117f10 con 0x7fa230070300 2023-06-23T01:18:23.244 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.238+0000 7fa2349ff700 1 -- 172.21.15.120:0/3142370421 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa230103430 con 0x7fa230070300 2023-06-23T01:18:23.244 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.240+0000 7fa21effd700 1 -- 172.21.15.120:0/3142370421 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7fa2200090d0 con 0x7fa230070300 2023-06-23T01:18:23.244 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.241+0000 7fa21effd700 1 --2- 172.21.15.120:0/3142370421 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fa21809b680 0x7fa21809db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:23.244 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.241+0000 7fa21effd700 1 -- 172.21.15.120:0/3142370421 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7fa2200c4530 con 0x7fa230070300 2023-06-23T01:18:23.244 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.241+0000 7fa22dd9b700 1 --2- 172.21.15.120:0/3142370421 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fa21809b680 0x7fa21809db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:23.245 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.241+0000 7fa22dd9b700 1 --2- 172.21.15.120:0/3142370421 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fa21809b680 0x7fa21809db30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa224009fd0 tx=0x7fa22400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:23.245 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.243+0000 7fa21effd700 1 -- 172.21.15.120:0/3142370421 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fa22004da00 con 0x7fa230070300 2023-06-23T01:18:23.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:22 smithi120 ceph-mon[79813]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:23.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:22 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:23.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:22 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:23.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:22 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:23.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:22 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T01:18:23.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:22 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T01:18:23.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:22 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:18:23.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:22 smithi157 ceph-mon[116945]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:23.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:22 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:23.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:22 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:23.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:22 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:23.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:22 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T01:18:23.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:22 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T01:18:23.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:22 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:18:23.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:22 smithi134 ceph-mon[116146]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:23.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:22 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:23.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:22 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:23.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:22 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:23.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:22 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T01:18:23.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:22 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T01:18:23.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:22 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:18:23.502 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.499+0000 7fa2349ff700 1 -- 172.21.15.120:0/3142370421 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa23002afc0 con 0x7fa230070300 2023-06-23T01:18:23.503 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.500+0000 7fa21effd700 1 -- 172.21.15.120:0/3142370421 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7342 (secure 0 0 0) 0x7fa220019090 con 0x7fa230070300 2023-06-23T01:18:23.503 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:18:23.503 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14427,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1538161586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1538161586},{"type":"v1","addr":"172.21.15.120:6835","nonce":1538161586}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/2588400819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":2588400819},{"type":"v1","addr":"172.21.15.120:6837","nonce":2588400819}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3214085078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3214085078},{"type":"v1","addr":"172.21.15.134:6833","nonce":3214085078}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1478747127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1478747127},{"type":"v1","addr":"172.21.15.157:6833","nonce":1478747127}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/4238446318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":4238446318},{"type":"v1","addr":"172.21.15.134:6835","nonce":4238446318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2129674435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2129674435},{"type":"v1","addr":"172.21.15.157:6835","nonce":2129674435}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3499504142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3499504142},{"type":"v1","addr":"172.21.15.134:6837","nonce":3499504142}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1606323168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1606323168},{"type":"v1","addr":"172.21.15.157:6837","nonce":1606323168}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24559,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/4251766478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4251766478},{"type":"v1","addr":"172.21.15.157:6839","nonce":4251766478}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-06-23T01:18:23.505 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.502+0000 7fa2349ff700 1 -- 172.21.15.120:0/3142370421 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fa21809b680 msgr2=0x7fa21809db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:23.506 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.502+0000 7fa2349ff700 1 --2- 172.21.15.120:0/3142370421 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fa21809b680 0x7fa21809db30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa224009fd0 tx=0x7fa22400b040 comp rx=0 tx=0).stop 2023-06-23T01:18:23.506 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.503+0000 7fa2349ff700 1 -- 172.21.15.120:0/3142370421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa230070300 msgr2=0x7fa23010f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:23.506 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.503+0000 7fa2349ff700 1 --2- 172.21.15.120:0/3142370421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa230070300 0x7fa23010f9a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa22000ebf0 tx=0x7fa22000b040 comp rx=0 tx=0).stop 2023-06-23T01:18:23.506 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.503+0000 7fa2349ff700 1 -- 172.21.15.120:0/3142370421 shutdown_connections 2023-06-23T01:18:23.506 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.503+0000 7fa2349ff700 1 --2- 172.21.15.120:0/3142370421 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fa21809b680 0x7fa21809db30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:23.506 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.503+0000 7fa2349ff700 1 --2- 172.21.15.120:0/3142370421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa230104ce0 0x7fa230116f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:23.506 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.503+0000 7fa2349ff700 1 --2- 172.21.15.120:0/3142370421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa230070300 0x7fa23010f9a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:23.507 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.503+0000 7fa2349ff700 1 --2- 172.21.15.120:0/3142370421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa230071820 0x7fa23010fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:23.507 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.503+0000 7fa2349ff700 1 -- 172.21.15.120:0/3142370421 >> 172.21.15.120:0/3142370421 conn(0x7fa2300f84a0 msgr2=0x7fa230106eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:23.507 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.503+0000 7fa2349ff700 1 -- 172.21.15.120:0/3142370421 shutdown_connections 2023-06-23T01:18:23.507 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:23.503+0000 7fa2349ff700 1 -- 172.21.15.120:0/3142370421 wait complete. 2023-06-23T01:18:23.507 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 10 2023-06-23T01:18:24.207 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:23 smithi120 ceph-mon[79813]: Deploying daemon mds.g on smithi120 2023-06-23T01:18:24.207 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:23 smithi120 ceph-mon[79813]: mds.? [v2:172.21.15.157:6838/4251766478,v1:172.21.15.157:6839/4251766478] up:boot 2023-06-23T01:18:24.207 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:23 smithi120 ceph-mon[79813]: fsmap 9 up:standby 2023-06-23T01:18:24.207 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:23 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-06-23T01:18:24.207 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:23 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/3142370421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:24.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:23 smithi157 ceph-mon[116945]: Deploying daemon mds.g on smithi120 2023-06-23T01:18:24.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:23 smithi157 ceph-mon[116945]: mds.? [v2:172.21.15.157:6838/4251766478,v1:172.21.15.157:6839/4251766478] up:boot 2023-06-23T01:18:24.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:23 smithi157 ceph-mon[116945]: fsmap 9 up:standby 2023-06-23T01:18:24.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:23 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-06-23T01:18:24.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:23 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/3142370421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:24.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:23 smithi134 ceph-mon[116146]: Deploying daemon mds.g on smithi120 2023-06-23T01:18:24.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:23 smithi134 ceph-mon[116146]: mds.? [v2:172.21.15.157:6838/4251766478,v1:172.21.15.157:6839/4251766478] up:boot 2023-06-23T01:18:24.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:23 smithi134 ceph-mon[116146]: fsmap 9 up:standby 2023-06-23T01:18:24.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:23 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-06-23T01:18:24.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:23 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/3142370421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:25.033 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:24 smithi120 ceph-mon[79813]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:25.033 INFO:journalctl@ceph.mds.g.smithi120.stdout:Jun 23 01:18:25 smithi120 systemd[1]: Starting Ceph mds.g for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:18:25.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:24 smithi157 ceph-mon[116945]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:25.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:24 smithi134 ceph-mon[116146]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:25.526 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T01:18:25.717 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.714+0000 7f9c7f59d700 1 -- 172.21.15.120:0/3252139280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c780fe6b0 msgr2=0x7f9c78104f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:25.718 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.714+0000 7f9c7f59d700 1 --2- 172.21.15.120:0/3252139280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c780fe6b0 0x7f9c78104f70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9c70009d60 tx=0x7f9c70004d10 comp rx=0 tx=0).stop 2023-06-23T01:18:25.718 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.714+0000 7f9c7f59d700 1 -- 172.21.15.120:0/3252139280 shutdown_connections 2023-06-23T01:18:25.718 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.714+0000 7f9c7f59d700 1 --2- 172.21.15.120:0/3252139280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c780fe6b0 0x7f9c78104f70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:25.718 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.714+0000 7f9c7f59d700 1 --2- 172.21.15.120:0/3252139280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c780fcb10 0x7f9c780fcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:25.718 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.714+0000 7f9c7f59d700 1 --2- 172.21.15.120:0/3252139280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c780fdd20 0x7f9c780fe170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:25.719 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.714+0000 7f9c7f59d700 1 -- 172.21.15.120:0/3252139280 >> 172.21.15.120:0/3252139280 conn(0x7f9c780f8230 msgr2=0x7f9c780fa6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:25.719 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.715+0000 7f9c7f59d700 1 -- 172.21.15.120:0/3252139280 shutdown_connections 2023-06-23T01:18:25.719 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.715+0000 7f9c7f59d700 1 -- 172.21.15.120:0/3252139280 wait complete. 2023-06-23T01:18:25.722 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.715+0000 7f9c7f59d700 1 Processor -- start 2023-06-23T01:18:25.722 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.715+0000 7f9c7f59d700 1 -- start start 2023-06-23T01:18:25.722 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.716+0000 7f9c7f59d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c780fcb10 0x7f9c7810f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:25.722 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.716+0000 7f9c7f59d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c780fdd20 0x7f9c7810fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:25.722 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.716+0000 7f9c7f59d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c780fe6b0 0x7f9c78116d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:25.722 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.716+0000 7f9c7f59d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c78117370 con 0x7f9c780fdd20 2023-06-23T01:18:25.723 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.716+0000 7f9c7f59d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c781174e0 con 0x7f9c780fcb10 2023-06-23T01:18:25.723 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.716+0000 7f9c7f59d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c781176a0 con 0x7f9c780fe6b0 2023-06-23T01:18:25.723 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.716+0000 7f9c7dd9a700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c780fdd20 0x7f9c7810fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:25.723 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.716+0000 7f9c7dd9a700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c780fdd20 0x7f9c7810fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57572/0 (socket says 172.21.15.120:57572) 2023-06-23T01:18:25.723 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.716+0000 7f9c7dd9a700 1 -- 172.21.15.120:0/473965568 learned_addr learned my addr 172.21.15.120:0/473965568 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:18:25.723 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.716+0000 7f9c7ed9c700 1 --2- 172.21.15.120:0/473965568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c780fe6b0 0x7f9c78116d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:25.723 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.716+0000 7f9c7e59b700 1 --2- 172.21.15.120:0/473965568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c780fcb10 0x7f9c7810f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:25.723 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.716+0000 7f9c7dd9a700 1 -- 172.21.15.120:0/473965568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c780fe6b0 msgr2=0x7f9c78116d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:25.724 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.716+0000 7f9c7dd9a700 1 --2- 172.21.15.120:0/473965568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c780fe6b0 0x7f9c78116d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:25.724 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.716+0000 7f9c7dd9a700 1 -- 172.21.15.120:0/473965568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c780fcb10 msgr2=0x7f9c7810f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:25.724 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.716+0000 7f9c7dd9a700 1 --2- 172.21.15.120:0/473965568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c780fcb10 0x7f9c7810f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:25.724 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.716+0000 7f9c7dd9a700 1 -- 172.21.15.120:0/473965568 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c70009940 con 0x7f9c780fdd20 2023-06-23T01:18:25.724 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.716+0000 7f9c7dd9a700 1 --2- 172.21.15.120:0/473965568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c780fdd20 0x7f9c7810fcd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9c6800ebc0 tx=0x7f9c6800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:25.724 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.717+0000 7f9c6effd700 1 -- 172.21.15.120:0/473965568 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c68007cc0 con 0x7f9c780fdd20 2023-06-23T01:18:25.724 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.717+0000 7f9c6effd700 1 -- 172.21.15.120:0/473965568 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c6800f040 con 0x7f9c780fdd20 2023-06-23T01:18:25.725 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.717+0000 7f9c7f59d700 1 -- 172.21.15.120:0/473965568 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c78117920 con 0x7f9c780fdd20 2023-06-23T01:18:25.725 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.717+0000 7f9c6effd700 1 -- 172.21.15.120:0/473965568 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c680094a0 con 0x7f9c780fdd20 2023-06-23T01:18:25.725 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.717+0000 7f9c7f59d700 1 -- 172.21.15.120:0/473965568 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c78117e90 con 0x7f9c780fdd20 2023-06-23T01:18:25.725 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.718+0000 7f9c7f59d700 1 -- 172.21.15.120:0/473965568 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c78108360 con 0x7f9c780fdd20 2023-06-23T01:18:25.726 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.720+0000 7f9c6effd700 1 -- 172.21.15.120:0/473965568 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7f9c6800c600 con 0x7f9c780fdd20 2023-06-23T01:18:25.726 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.723+0000 7f9c6effd700 1 --2- 172.21.15.120:0/473965568 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f9c6409b680 0x7f9c6409db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:25.726 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.723+0000 7f9c6effd700 1 -- 172.21.15.120:0/473965568 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f9c680c45c0 con 0x7f9c780fdd20 2023-06-23T01:18:25.726 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.723+0000 7f9c7e59b700 1 --2- 172.21.15.120:0/473965568 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f9c6409b680 0x7f9c6409db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:25.727 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.723+0000 7f9c7e59b700 1 --2- 172.21.15.120:0/473965568 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f9c6409b680 0x7f9c6409db30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9c74004600 tx=0x7f9c7400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:25.729 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:25.726+0000 7f9c6effd700 1 -- 172.21.15.120:0/473965568 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f9c6804da00 con 0x7f9c780fdd20 2023-06-23T01:18:26.004 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:26.001+0000 7f9c7f59d700 1 -- 172.21.15.120:0/473965568 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c78118080 con 0x7f9c780fdd20 2023-06-23T01:18:26.004 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:26.001+0000 7f9c6effd700 1 -- 172.21.15.120:0/473965568 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7342 (secure 0 0 0) 0x7f9c68090220 con 0x7f9c780fdd20 2023-06-23T01:18:26.005 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:18:26.005 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14427,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1538161586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1538161586},{"type":"v1","addr":"172.21.15.120:6835","nonce":1538161586}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/2588400819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":2588400819},{"type":"v1","addr":"172.21.15.120:6837","nonce":2588400819}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3214085078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3214085078},{"type":"v1","addr":"172.21.15.134:6833","nonce":3214085078}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1478747127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1478747127},{"type":"v1","addr":"172.21.15.157:6833","nonce":1478747127}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/4238446318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":4238446318},{"type":"v1","addr":"172.21.15.134:6835","nonce":4238446318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2129674435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2129674435},{"type":"v1","addr":"172.21.15.157:6835","nonce":2129674435}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3499504142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3499504142},{"type":"v1","addr":"172.21.15.134:6837","nonce":3499504142}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1606323168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1606323168},{"type":"v1","addr":"172.21.15.157:6837","nonce":1606323168}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24559,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/4251766478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4251766478},{"type":"v1","addr":"172.21.15.157:6839","nonce":4251766478}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-06-23T01:18:26.007 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:26.003+0000 7f9c7f59d700 1 -- 172.21.15.120:0/473965568 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f9c6409b680 msgr2=0x7f9c6409db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:26.007 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:26.003+0000 7f9c7f59d700 1 --2- 172.21.15.120:0/473965568 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f9c6409b680 0x7f9c6409db30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9c74004600 tx=0x7f9c7400b040 comp rx=0 tx=0).stop 2023-06-23T01:18:26.007 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:26.003+0000 7f9c7f59d700 1 -- 172.21.15.120:0/473965568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c780fdd20 msgr2=0x7f9c7810fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:26.007 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:26.003+0000 7f9c7f59d700 1 --2- 172.21.15.120:0/473965568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c780fdd20 0x7f9c7810fcd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9c6800ebc0 tx=0x7f9c6800b040 comp rx=0 tx=0).stop 2023-06-23T01:18:26.007 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:26.004+0000 7f9c7f59d700 1 -- 172.21.15.120:0/473965568 shutdown_connections 2023-06-23T01:18:26.008 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:26.004+0000 7f9c7f59d700 1 --2- 172.21.15.120:0/473965568 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f9c6409b680 0x7f9c6409db30 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:26.008 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:26.004+0000 7f9c7f59d700 1 --2- 172.21.15.120:0/473965568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c780fe6b0 0x7f9c78116d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:26.008 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:26.004+0000 7f9c7f59d700 1 --2- 172.21.15.120:0/473965568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c780fcb10 0x7f9c7810f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:26.008 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:26.004+0000 7f9c7f59d700 1 --2- 172.21.15.120:0/473965568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c780fdd20 0x7f9c7810fcd0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:26.008 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:26.004+0000 7f9c7f59d700 1 -- 172.21.15.120:0/473965568 >> 172.21.15.120:0/473965568 conn(0x7f9c780f8230 msgr2=0x7f9c780fa190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:26.008 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:26.004+0000 7f9c7f59d700 1 -- 172.21.15.120:0/473965568 shutdown_connections 2023-06-23T01:18:26.008 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:26.004+0000 7f9c7f59d700 1 -- 172.21.15.120:0/473965568 wait complete. 2023-06-23T01:18:26.009 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 10 2023-06-23T01:18:26.510 INFO:journalctl@ceph.mds.g.smithi120.stdout:Jun 23 01:18:26 smithi120 bash[124010]: 5f665ef8913043bfc2ef4c3286d53b0fd0d7b32a2182452841616768a6ab0aa3 2023-06-23T01:18:26.510 INFO:journalctl@ceph.mds.g.smithi120.stdout:Jun 23 01:18:26 smithi120 ceph-mds[124116]: -- 172.21.15.120:0/3443257004 <== mon.0 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55db321f89c0 con 0x55db321cec00 2023-06-23T01:18:26.511 INFO:journalctl@ceph.mds.g.smithi120.stdout:Jun 23 01:18:26 smithi120 conmon[124089]: starting mds.g at 2023-06-23T01:18:26.511 INFO:journalctl@ceph.mds.g.smithi120.stdout:Jun 23 01:18:26 smithi120 systemd[1]: Started Ceph mds.g for 9efbfa8e-1162-11ee-9b2b-001a4aab830c. 2023-06-23T01:18:27.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:26 smithi120 ceph-mon[79813]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:27.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:26 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/473965568' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:27.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:26 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:27.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:26 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:27.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:26 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:27.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:26 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T01:18:27.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:26 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T01:18:27.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:26 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:18:27.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:26 smithi157 ceph-mon[116945]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:27.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:26 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/473965568' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:27.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:26 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:27.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:26 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:27.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:26 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:27.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:26 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T01:18:27.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:26 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T01:18:27.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:26 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:18:27.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:26 smithi134 ceph-mon[116146]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:27.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:26 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/473965568' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:27.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:26 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:27.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:26 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:27.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:26 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:27.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:26 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T01:18:27.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:26 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T01:18:27.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:26 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:18:28.025 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T01:18:28.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:27 smithi120 ceph-mon[79813]: Deploying daemon mds.a on smithi120 2023-06-23T01:18:28.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:27 smithi120 ceph-mon[79813]: mds.? [v2:172.21.15.120:6838/1756726791,v1:172.21.15.120:6839/1756726791] up:boot 2023-06-23T01:18:28.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:27 smithi120 ceph-mon[79813]: fsmap 10 up:standby 2023-06-23T01:18:28.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:27 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-06-23T01:18:28.276 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.272+0000 7f52ccf9d700 1 -- 172.21.15.120:0/2427872386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52c80fdd20 msgr2=0x7f52c80fe170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:28.276 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.273+0000 7f52ccf9d700 1 --2- 172.21.15.120:0/2427872386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52c80fdd20 0x7f52c80fe170 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f52bc0049c0 tx=0x7f52bc009e30 comp rx=0 tx=0).stop 2023-06-23T01:18:28.276 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.273+0000 7f52ccf9d700 1 -- 172.21.15.120:0/2427872386 shutdown_connections 2023-06-23T01:18:28.276 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.273+0000 7f52ccf9d700 1 --2- 172.21.15.120:0/2427872386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52c80fcb10 0x7f52c80fcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:28.277 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.273+0000 7f52ccf9d700 1 --2- 172.21.15.120:0/2427872386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52c80fe6b0 0x7f52c8104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:28.277 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.273+0000 7f52ccf9d700 1 --2- 172.21.15.120:0/2427872386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52c80fdd20 0x7f52c80fe170 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:28.277 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.273+0000 7f52ccf9d700 1 -- 172.21.15.120:0/2427872386 >> 172.21.15.120:0/2427872386 conn(0x7f52c80f8230 msgr2=0x7f52c80fa6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:28.277 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.273+0000 7f52ccf9d700 1 -- 172.21.15.120:0/2427872386 shutdown_connections 2023-06-23T01:18:28.277 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.273+0000 7f52ccf9d700 1 -- 172.21.15.120:0/2427872386 wait complete. 2023-06-23T01:18:28.279 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.273+0000 7f52ccf9d700 1 Processor -- start 2023-06-23T01:18:28.279 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.274+0000 7f52ccf9d700 1 -- start start 2023-06-23T01:18:28.280 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.274+0000 7f52ccf9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52c80fcb10 0x7f52c810f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:28.280 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.274+0000 7f52ccf9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52c80fdd20 0x7f52c810ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:28.280 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.274+0000 7f52ccf9d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52c80fe6b0 0x7f52c8116fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:28.280 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.274+0000 7f52ccf9d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c81175c0 con 0x7f52c80fe6b0 2023-06-23T01:18:28.280 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.274+0000 7f52ccf9d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c8117730 con 0x7f52c80fdd20 2023-06-23T01:18:28.280 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.274+0000 7f52ccf9d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c81178f0 con 0x7f52c80fcb10 2023-06-23T01:18:28.280 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.274+0000 7f52c7fff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52c80fe6b0 0x7f52c8116fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:28.281 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.274+0000 7f52c7fff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52c80fe6b0 0x7f52c8116fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57606/0 (socket says 172.21.15.120:57606) 2023-06-23T01:18:28.281 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.274+0000 7f52c7fff700 1 -- 172.21.15.120:0/1918732767 learned_addr learned my addr 172.21.15.120:0/1918732767 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:18:28.281 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.274+0000 7f52c6ffd700 1 --2- 172.21.15.120:0/1918732767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52c80fdd20 0x7f52c810ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:28.281 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.274+0000 7f52c77fe700 1 --2- 172.21.15.120:0/1918732767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52c80fcb10 0x7f52c810f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:28.281 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.275+0000 7f52c6ffd700 1 -- 172.21.15.120:0/1918732767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52c80fcb10 msgr2=0x7f52c810f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:28.281 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.275+0000 7f52c6ffd700 1 --2- 172.21.15.120:0/1918732767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52c80fcb10 0x7f52c810f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:28.281 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.275+0000 7f52c6ffd700 1 -- 172.21.15.120:0/1918732767 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52c80fe6b0 msgr2=0x7f52c8116fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:28.281 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.275+0000 7f52c6ffd700 1 --2- 172.21.15.120:0/1918732767 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52c80fe6b0 0x7f52c8116fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:28.282 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.275+0000 7f52c6ffd700 1 -- 172.21.15.120:0/1918732767 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b8005040 con 0x7f52c80fdd20 2023-06-23T01:18:28.282 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.275+0000 7f52c7fff700 1 --2- 172.21.15.120:0/1918732767 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52c80fe6b0 0x7f52c8116fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T01:18:28.282 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.275+0000 7f52c6ffd700 1 --2- 172.21.15.120:0/1918732767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52c80fdd20 0x7f52c810ff20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f52bc0049c0 tx=0x7f52bc009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:28.282 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.275+0000 7f52affff700 1 -- 172.21.15.120:0/1918732767 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc00e070 con 0x7f52c80fdd20 2023-06-23T01:18:28.282 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.275+0000 7f52ccf9d700 1 -- 172.21.15.120:0/1918732767 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52bc005040 con 0x7f52c80fdd20 2023-06-23T01:18:28.282 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.275+0000 7f52affff700 1 -- 172.21.15.120:0/1918732767 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52bc009870 con 0x7f52c80fdd20 2023-06-23T01:18:28.282 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.275+0000 7f52affff700 1 -- 172.21.15.120:0/1918732767 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc0123c0 con 0x7f52c80fdd20 2023-06-23T01:18:28.282 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.275+0000 7f52ccf9d700 1 -- 172.21.15.120:0/1918732767 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c8117f10 con 0x7f52c80fdd20 2023-06-23T01:18:28.283 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.276+0000 7f52ccf9d700 1 -- 172.21.15.120:0/1918732767 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52c8108350 con 0x7f52c80fdd20 2023-06-23T01:18:28.283 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.278+0000 7f52affff700 1 -- 172.21.15.120:0/1918732767 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7f52bc04bbb0 con 0x7f52c80fdd20 2023-06-23T01:18:28.284 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.280+0000 7f52affff700 1 --2- 172.21.15.120:0/1918732767 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f52b009b680 0x7f52b009db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:28.284 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.280+0000 7f52affff700 1 -- 172.21.15.120:0/1918732767 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f52bc0c5ce0 con 0x7f52c80fdd20 2023-06-23T01:18:28.284 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.280+0000 7f52c77fe700 1 --2- 172.21.15.120:0/1918732767 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f52b009b680 0x7f52b009db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:28.284 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.280+0000 7f52c77fe700 1 --2- 172.21.15.120:0/1918732767 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f52b009b680 0x7f52b009db30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f52c80fdb80 tx=0x7f52b800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:28.284 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.282+0000 7f52affff700 1 -- 172.21.15.120:0/1918732767 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f52bc091940 con 0x7f52c80fdd20 2023-06-23T01:18:28.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:27 smithi157 ceph-mon[116945]: Deploying daemon mds.a on smithi120 2023-06-23T01:18:28.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:27 smithi157 ceph-mon[116945]: mds.? [v2:172.21.15.120:6838/1756726791,v1:172.21.15.120:6839/1756726791] up:boot 2023-06-23T01:18:28.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:27 smithi157 ceph-mon[116945]: fsmap 10 up:standby 2023-06-23T01:18:28.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:27 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-06-23T01:18:28.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:27 smithi134 ceph-mon[116146]: Deploying daemon mds.a on smithi120 2023-06-23T01:18:28.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:27 smithi134 ceph-mon[116146]: mds.? [v2:172.21.15.120:6838/1756726791,v1:172.21.15.120:6839/1756726791] up:boot 2023-06-23T01:18:28.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:27 smithi134 ceph-mon[116146]: fsmap 10 up:standby 2023-06-23T01:18:28.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:27 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-06-23T01:18:28.538 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.535+0000 7f52ccf9d700 1 -- 172.21.15.120:0/1918732767 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52c81084e0 con 0x7f52c80fdd20 2023-06-23T01:18:28.539 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.536+0000 7f52affff700 1 -- 172.21.15.120:0/1918732767 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8103 (secure 0 0 0) 0x7f52c81084e0 con 0x7f52c80fdd20 2023-06-23T01:18:28.539 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:18:28.540 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14427,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1538161586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1538161586},{"type":"v1","addr":"172.21.15.120:6835","nonce":1538161586}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/2588400819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":2588400819},{"type":"v1","addr":"172.21.15.120:6837","nonce":2588400819}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/1756726791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1756726791},{"type":"v1","addr":"172.21.15.120:6839","nonce":1756726791}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3214085078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3214085078},{"type":"v1","addr":"172.21.15.134:6833","nonce":3214085078}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1478747127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1478747127},{"type":"v1","addr":"172.21.15.157:6833","nonce":1478747127}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/4238446318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":4238446318},{"type":"v1","addr":"172.21.15.134:6835","nonce":4238446318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2129674435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2129674435},{"type":"v1","addr":"172.21.15.157:6835","nonce":2129674435}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3499504142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3499504142},{"type":"v1","addr":"172.21.15.134:6837","nonce":3499504142}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1606323168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1606323168},{"type":"v1","addr":"172.21.15.157:6837","nonce":1606323168}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24559,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/4251766478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4251766478},{"type":"v1","addr":"172.21.15.157:6839","nonce":4251766478}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-06-23T01:18:28.541 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.538+0000 7f52ccf9d700 1 -- 172.21.15.120:0/1918732767 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f52b009b680 msgr2=0x7f52b009db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:28.542 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.538+0000 7f52ccf9d700 1 --2- 172.21.15.120:0/1918732767 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f52b009b680 0x7f52b009db30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f52c80fdb80 tx=0x7f52b800b040 comp rx=0 tx=0).stop 2023-06-23T01:18:28.542 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.538+0000 7f52ccf9d700 1 -- 172.21.15.120:0/1918732767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52c80fdd20 msgr2=0x7f52c810ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:28.542 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.538+0000 7f52ccf9d700 1 --2- 172.21.15.120:0/1918732767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52c80fdd20 0x7f52c810ff20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f52bc0049c0 tx=0x7f52bc009e30 comp rx=0 tx=0).stop 2023-06-23T01:18:28.542 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.539+0000 7f52ccf9d700 1 -- 172.21.15.120:0/1918732767 shutdown_connections 2023-06-23T01:18:28.542 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.539+0000 7f52ccf9d700 1 --2- 172.21.15.120:0/1918732767 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f52b009b680 0x7f52b009db30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:28.542 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.539+0000 7f52ccf9d700 1 --2- 172.21.15.120:0/1918732767 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52c80fe6b0 0x7f52c8116fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:28.543 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.539+0000 7f52ccf9d700 1 --2- 172.21.15.120:0/1918732767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52c80fcb10 0x7f52c810f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:28.543 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.539+0000 7f52ccf9d700 1 --2- 172.21.15.120:0/1918732767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52c80fdd20 0x7f52c810ff20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:28.543 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.539+0000 7f52ccf9d700 1 -- 172.21.15.120:0/1918732767 >> 172.21.15.120:0/1918732767 conn(0x7f52c80f8230 msgr2=0x7f52c80fa480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:28.543 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.539+0000 7f52ccf9d700 1 -- 172.21.15.120:0/1918732767 shutdown_connections 2023-06-23T01:18:28.543 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:28.539+0000 7f52ccf9d700 1 -- 172.21.15.120:0/1918732767 wait complete. 2023-06-23T01:18:28.544 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 11 2023-06-23T01:18:28.760 INFO:journalctl@ceph.mds.a.smithi120.stdout:Jun 23 01:18:28 smithi120 systemd[1]: Starting Ceph mds.a for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:18:29.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:28 smithi120 ceph-mon[79813]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:29.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:28 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/1918732767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:29.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:28 smithi157 ceph-mon[116945]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:29.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:28 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/1918732767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:29.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:28 smithi134 ceph-mon[116146]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:29.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:28 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/1918732767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:29.760 INFO:journalctl@ceph.mds.a.smithi120.stdout:Jun 23 01:18:29 smithi120 bash[124658]: e88e6be5f3f1276250abbd3ca1cc3c7554eb0ef35db9633dbd93fc7230bd52d5 2023-06-23T01:18:29.761 INFO:journalctl@ceph.mds.a.smithi120.stdout:Jun 23 01:18:29 smithi120 ceph-mds[124723]: -- 172.21.15.120:0/749071154 <== mon.0 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55688bf389c0 con 0x55688bf0ec00 2023-06-23T01:18:29.761 INFO:journalctl@ceph.mds.a.smithi120.stdout:Jun 23 01:18:29 smithi120 conmon[124697]: starting mds.a at 2023-06-23T01:18:29.761 INFO:journalctl@ceph.mds.a.smithi120.stdout:Jun 23 01:18:29 smithi120 systemd[1]: Started Ceph mds.a for 9efbfa8e-1162-11ee-9b2b-001a4aab830c. 2023-06-23T01:18:30.563 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T01:18:30.758 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.753+0000 7f9dc93ff700 1 -- 172.21.15.120:0/389088994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dc40fe930 msgr2=0x7f9dc41051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:30.758 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.753+0000 7f9dc93ff700 1 --2- 172.21.15.120:0/389088994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dc40fe930 0x7f9dc41051d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc009d60 tx=0x7f9dbc004d10 comp rx=0 tx=0).stop 2023-06-23T01:18:30.759 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.754+0000 7f9dc93ff700 1 -- 172.21.15.120:0/389088994 shutdown_connections 2023-06-23T01:18:30.759 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.754+0000 7f9dc93ff700 1 --2- 172.21.15.120:0/389088994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dc40fe930 0x7f9dc41051d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:30.759 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.754+0000 7f9dc93ff700 1 --2- 172.21.15.120:0/389088994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dc40fccd0 0x7f9dc40fd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:30.759 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.754+0000 7f9dc93ff700 1 --2- 172.21.15.120:0/389088994 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9dc40fdf80 0x7f9dc40fe3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:30.759 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.754+0000 7f9dc93ff700 1 -- 172.21.15.120:0/389088994 >> 172.21.15.120:0/389088994 conn(0x7f9dc40f83f0 msgr2=0x7f9dc40fa860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:30.759 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.754+0000 7f9dc93ff700 1 -- 172.21.15.120:0/389088994 shutdown_connections 2023-06-23T01:18:30.759 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.754+0000 7f9dc93ff700 1 -- 172.21.15.120:0/389088994 wait complete. 2023-06-23T01:18:30.760 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.754+0000 7f9dc93ff700 1 Processor -- start 2023-06-23T01:18:30.760 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.755+0000 7f9dc93ff700 1 -- start start 2023-06-23T01:18:30.760 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.755+0000 7f9dc93ff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9dc40fccd0 0x7f9dc410d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:30.760 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.755+0000 7f9dc93ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dc40fdf80 0x7f9dc410dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:30.760 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.755+0000 7f9dc93ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dc40fe930 0x7f9dc4114d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:30.760 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.755+0000 7f9dc93ff700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc4115330 con 0x7f9dc40fccd0 2023-06-23T01:18:30.760 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.755+0000 7f9dc93ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc41154a0 con 0x7f9dc40fe930 2023-06-23T01:18:30.760 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.755+0000 7f9dc93ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc4115660 con 0x7f9dc40fdf80 2023-06-23T01:18:30.761 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.755+0000 7f9dc2ffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9dc40fccd0 0x7f9dc410d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:30.761 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.755+0000 7f9dc27fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dc40fdf80 0x7f9dc410dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:30.761 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.755+0000 7f9dc2ffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9dc40fccd0 0x7f9dc410d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57648/0 (socket says 172.21.15.120:57648) 2023-06-23T01:18:30.761 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.755+0000 7f9dc2ffd700 1 -- 172.21.15.120:0/1923897829 learned_addr learned my addr 172.21.15.120:0/1923897829 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:18:30.761 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.755+0000 7f9dc37fe700 1 --2- 172.21.15.120:0/1923897829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dc40fe930 0x7f9dc4114d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:30.762 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.755+0000 7f9dc2ffd700 1 -- 172.21.15.120:0/1923897829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dc40fdf80 msgr2=0x7f9dc410dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:30.763 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.755+0000 7f9dc2ffd700 1 --2- 172.21.15.120:0/1923897829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dc40fdf80 0x7f9dc410dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:30.763 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.755+0000 7f9dc2ffd700 1 -- 172.21.15.120:0/1923897829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dc40fe930 msgr2=0x7f9dc4114d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:30.763 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.755+0000 7f9dc2ffd700 1 --2- 172.21.15.120:0/1923897829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dc40fe930 0x7f9dc4114d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:30.763 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.755+0000 7f9dc2ffd700 1 -- 172.21.15.120:0/1923897829 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dbc009940 con 0x7f9dc40fccd0 2023-06-23T01:18:30.763 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.756+0000 7f9dc2ffd700 1 --2- 172.21.15.120:0/1923897829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9dc40fccd0 0x7f9dc410d750 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9db400ebc0 tx=0x7f9db400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:30.763 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.756+0000 7f9dab7fe700 1 -- 172.21.15.120:0/1923897829 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db4007cc0 con 0x7f9dc40fccd0 2023-06-23T01:18:30.764 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.756+0000 7f9dab7fe700 1 -- 172.21.15.120:0/1923897829 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9db400f040 con 0x7f9dc40fccd0 2023-06-23T01:18:30.764 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.756+0000 7f9dc93ff700 1 -- 172.21.15.120:0/1923897829 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc41158e0 con 0x7f9dc40fccd0 2023-06-23T01:18:30.764 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.756+0000 7f9dab7fe700 1 -- 172.21.15.120:0/1923897829 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db400c880 con 0x7f9dc40fccd0 2023-06-23T01:18:30.764 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.756+0000 7f9dc93ff700 1 -- 172.21.15.120:0/1923897829 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc4115e50 con 0x7f9dc40fccd0 2023-06-23T01:18:30.764 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.757+0000 7f9dc93ff700 1 -- 172.21.15.120:0/1923897829 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc4108530 con 0x7f9dc40fccd0 2023-06-23T01:18:30.764 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.759+0000 7f9dab7fe700 1 -- 172.21.15.120:0/1923897829 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7f9db400ccb0 con 0x7f9dc40fccd0 2023-06-23T01:18:30.765 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.761+0000 7f9dab7fe700 1 --2- 172.21.15.120:0/1923897829 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f9dac09b680 0x7f9dac09db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:30.765 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.761+0000 7f9dab7fe700 1 -- 172.21.15.120:0/1923897829 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f9db40c4470 con 0x7f9dc40fccd0 2023-06-23T01:18:30.765 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.761+0000 7f9dc27fc700 1 --2- 172.21.15.120:0/1923897829 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f9dac09b680 0x7f9dac09db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:30.766 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.762+0000 7f9dc27fc700 1 --2- 172.21.15.120:0/1923897829 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f9dac09b680 0x7f9dac09db30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9db8009b80 tx=0x7f9db800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:30.768 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:30.765+0000 7f9dab7fe700 1 -- 172.21.15.120:0/1923897829 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f9db404da00 con 0x7f9dc40fccd0 2023-06-23T01:18:30.876 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:30 smithi157 ceph-mon[116945]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:30.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:30 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:30.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:30 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:30.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:30 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:30.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:30 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T01:18:30.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:30 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T01:18:30.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:30 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:18:30.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:30 smithi157 ceph-mon[116945]: Deploying daemon mds.b on smithi134 2023-06-23T01:18:30.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:30 smithi157 ceph-mon[116945]: mds.? [v2:172.21.15.120:6840/2429971001,v1:172.21.15.120:6841/2429971001] up:boot 2023-06-23T01:18:30.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:30 smithi157 ceph-mon[116945]: fsmap 11 up:standby 2023-06-23T01:18:30.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:30 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-06-23T01:18:30.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:30 smithi134 ceph-mon[116146]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:30.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:30 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:30.944 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:30 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:30.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:30 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:30.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:30 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T01:18:30.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:30 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T01:18:30.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:30 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:18:30.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:30 smithi134 ceph-mon[116146]: Deploying daemon mds.b on smithi134 2023-06-23T01:18:30.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:30 smithi134 ceph-mon[116146]: mds.? [v2:172.21.15.120:6840/2429971001,v1:172.21.15.120:6841/2429971001] up:boot 2023-06-23T01:18:30.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:30 smithi134 ceph-mon[116146]: fsmap 11 up:standby 2023-06-23T01:18:30.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:30 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-06-23T01:18:31.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:30 smithi120 ceph-mon[79813]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:31.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:30 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:31.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:30 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:31.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:30 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:31.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:30 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T01:18:31.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:30 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T01:18:31.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:30 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:18:31.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:30 smithi120 ceph-mon[79813]: Deploying daemon mds.b on smithi134 2023-06-23T01:18:31.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:30 smithi120 ceph-mon[79813]: mds.? [v2:172.21.15.120:6840/2429971001,v1:172.21.15.120:6841/2429971001] up:boot 2023-06-23T01:18:31.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:30 smithi120 ceph-mon[79813]: fsmap 11 up:standby 2023-06-23T01:18:31.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:30 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-06-23T01:18:31.017 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:31.014+0000 7f9dc93ff700 1 -- 172.21.15.120:0/1923897829 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9dc4116040 con 0x7f9dc40fccd0 2023-06-23T01:18:31.018 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:31.015+0000 7f9dab7fe700 1 -- 172.21.15.120:0/1923897829 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8864 (secure 0 0 0) 0x7f9db4090170 con 0x7f9dc40fccd0 2023-06-23T01:18:31.018 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:18:31.019 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14427,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1538161586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1538161586},{"type":"v1","addr":"172.21.15.120:6835","nonce":1538161586}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/2588400819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":2588400819},{"type":"v1","addr":"172.21.15.120:6837","nonce":2588400819}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/1756726791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1756726791},{"type":"v1","addr":"172.21.15.120:6839","nonce":1756726791}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/2429971001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":2429971001},{"type":"v1","addr":"172.21.15.120:6841","nonce":2429971001}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3214085078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3214085078},{"type":"v1","addr":"172.21.15.134:6833","nonce":3214085078}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1478747127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1478747127},{"type":"v1","addr":"172.21.15.157:6833","nonce":1478747127}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/4238446318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":4238446318},{"type":"v1","addr":"172.21.15.134:6835","nonce":4238446318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2129674435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2129674435},{"type":"v1","addr":"172.21.15.157:6835","nonce":2129674435}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3499504142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3499504142},{"type":"v1","addr":"172.21.15.134:6837","nonce":3499504142}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1606323168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1606323168},{"type":"v1","addr":"172.21.15.157:6837","nonce":1606323168}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24559,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/4251766478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4251766478},{"type":"v1","addr":"172.21.15.157:6839","nonce":4251766478}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-06-23T01:18:31.020 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:31.017+0000 7f9dc93ff700 1 -- 172.21.15.120:0/1923897829 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f9dac09b680 msgr2=0x7f9dac09db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:31.020 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:31.017+0000 7f9dc93ff700 1 --2- 172.21.15.120:0/1923897829 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f9dac09b680 0x7f9dac09db30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9db8009b80 tx=0x7f9db800b040 comp rx=0 tx=0).stop 2023-06-23T01:18:31.021 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:31.017+0000 7f9dc93ff700 1 -- 172.21.15.120:0/1923897829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9dc40fccd0 msgr2=0x7f9dc410d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:31.021 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:31.017+0000 7f9dc93ff700 1 --2- 172.21.15.120:0/1923897829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9dc40fccd0 0x7f9dc410d750 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9db400ebc0 tx=0x7f9db400b040 comp rx=0 tx=0).stop 2023-06-23T01:18:31.021 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:31.017+0000 7f9dc93ff700 1 -- 172.21.15.120:0/1923897829 shutdown_connections 2023-06-23T01:18:31.021 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:31.017+0000 7f9dc93ff700 1 --2- 172.21.15.120:0/1923897829 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f9dac09b680 0x7f9dac09db30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:31.021 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:31.017+0000 7f9dc93ff700 1 --2- 172.21.15.120:0/1923897829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9dc40fccd0 0x7f9dc410d750 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:31.021 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:31.017+0000 7f9dc93ff700 1 --2- 172.21.15.120:0/1923897829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dc40fe930 0x7f9dc4114d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:31.021 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:31.017+0000 7f9dc93ff700 1 --2- 172.21.15.120:0/1923897829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dc40fdf80 0x7f9dc410dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:31.022 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:31.017+0000 7f9dc93ff700 1 -- 172.21.15.120:0/1923897829 >> 172.21.15.120:0/1923897829 conn(0x7f9dc40f83f0 msgr2=0x7f9dc40fa3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:31.022 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:31.018+0000 7f9dc93ff700 1 -- 172.21.15.120:0/1923897829 shutdown_connections 2023-06-23T01:18:31.022 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:31.018+0000 7f9dc93ff700 1 -- 172.21.15.120:0/1923897829 wait complete. 2023-06-23T01:18:31.022 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 12 2023-06-23T01:18:31.834 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:31 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/1923897829' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:31.834 INFO:journalctl@ceph.mds.b.smithi134.stdout:Jun 23 01:18:31 smithi134 systemd[1]: Starting Ceph mds.b for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:18:31.877 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:31 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/1923897829' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:32.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:31 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/1923897829' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:32.444 INFO:journalctl@ceph.mds.b.smithi134.stdout:Jun 23 01:18:32 smithi134 ceph-mds[148956]: -- 172.21.15.134:0/2373742816 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5649cdb2a9c0 con 0x5649cdb00c00 2023-06-23T01:18:32.445 INFO:journalctl@ceph.mds.b.smithi134.stdout:Jun 23 01:18:32 smithi134 conmon[148932]: starting mds.b at 2023-06-23T01:18:32.445 INFO:journalctl@ceph.mds.b.smithi134.stdout:Jun 23 01:18:32 smithi134 bash[148892]: 06dd75b3aeebd181d5a664870ba4fbf56a3a91f45297be8379a1b09c3ae17bc8 2023-06-23T01:18:32.445 INFO:journalctl@ceph.mds.b.smithi134.stdout:Jun 23 01:18:32 smithi134 systemd[1]: Started Ceph mds.b for 9efbfa8e-1162-11ee-9b2b-001a4aab830c. 2023-06-23T01:18:33.040 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T01:18:33.238 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.235+0000 7f4b6579d700 1 -- 172.21.15.120:0/3499495148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b600fcb10 msgr2=0x7f4b600fcf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:33.238 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.235+0000 7f4b6579d700 1 --2- 172.21.15.120:0/3499495148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b600fcb10 0x7f4b600fcf20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4b500049c0 tx=0x7f4b50009e30 comp rx=0 tx=0).stop 2023-06-23T01:18:33.239 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.235+0000 7f4b6579d700 1 -- 172.21.15.120:0/3499495148 shutdown_connections 2023-06-23T01:18:33.239 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.235+0000 7f4b6579d700 1 --2- 172.21.15.120:0/3499495148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b600fcb10 0x7f4b600fcf20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:33.239 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.235+0000 7f4b6579d700 1 --2- 172.21.15.120:0/3499495148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b600fe6b0 0x7f4b60104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:33.239 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.235+0000 7f4b6579d700 1 --2- 172.21.15.120:0/3499495148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b600fdd20 0x7f4b600fe170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:33.239 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.235+0000 7f4b6579d700 1 -- 172.21.15.120:0/3499495148 >> 172.21.15.120:0/3499495148 conn(0x7f4b600f8230 msgr2=0x7f4b600fa6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:33.239 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.235+0000 7f4b6579d700 1 -- 172.21.15.120:0/3499495148 shutdown_connections 2023-06-23T01:18:33.240 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.235+0000 7f4b6579d700 1 -- 172.21.15.120:0/3499495148 wait complete. 2023-06-23T01:18:33.240 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.235+0000 7f4b6579d700 1 Processor -- start 2023-06-23T01:18:33.242 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.236+0000 7f4b6579d700 1 -- start start 2023-06-23T01:18:33.243 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.236+0000 7f4b6579d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b600fcb10 0x7f4b6010d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:33.243 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.236+0000 7f4b6579d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b600fdd20 0x7f4b6010dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:33.243 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.236+0000 7f4b6579d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b600fe6b0 0x7f4b60114d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:33.243 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.236+0000 7f4b6579d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b60115330 con 0x7f4b600fcb10 2023-06-23T01:18:33.244 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.236+0000 7f4b6579d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b601154a0 con 0x7f4b600fe6b0 2023-06-23T01:18:33.244 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.236+0000 7f4b6579d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b60115660 con 0x7f4b600fdd20 2023-06-23T01:18:33.244 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.236+0000 7f4b5ffff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b600fcb10 0x7f4b6010d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:33.244 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.236+0000 7f4b5f7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b600fdd20 0x7f4b6010dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:33.245 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.236+0000 7f4b64f9c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b600fe6b0 0x7f4b60114d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:33.245 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.236+0000 7f4b5f7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b600fdd20 0x7f4b6010dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.120:49342/0 (socket says 172.21.15.120:49342) 2023-06-23T01:18:33.245 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.236+0000 7f4b5f7fe700 1 -- 172.21.15.120:0/2675809679 learned_addr learned my addr 172.21.15.120:0/2675809679 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:18:33.245 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.236+0000 7f4b5f7fe700 1 -- 172.21.15.120:0/2675809679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b600fe6b0 msgr2=0x7f4b60114d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:33.245 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.236+0000 7f4b5f7fe700 1 --2- 172.21.15.120:0/2675809679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b600fe6b0 0x7f4b60114d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:33.246 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.237+0000 7f4b5f7fe700 1 -- 172.21.15.120:0/2675809679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b600fcb10 msgr2=0x7f4b6010d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:33.246 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.237+0000 7f4b5f7fe700 1 --2- 172.21.15.120:0/2675809679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b600fcb10 0x7f4b6010d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:33.246 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.237+0000 7f4b5f7fe700 1 -- 172.21.15.120:0/2675809679 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b50005040 con 0x7f4b600fdd20 2023-06-23T01:18:33.246 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.237+0000 7f4b5ffff700 1 --2- 172.21.15.120:0/2675809679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b600fcb10 0x7f4b6010d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T01:18:33.246 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.237+0000 7f4b5f7fe700 1 --2- 172.21.15.120:0/2675809679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b600fdd20 0x7f4b6010dc90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4b5400ebc0 tx=0x7f4b5400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:33.247 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.237+0000 7f4b5cff9700 1 -- 172.21.15.120:0/2675809679 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b54007cc0 con 0x7f4b600fdd20 2023-06-23T01:18:33.247 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.237+0000 7f4b6579d700 1 -- 172.21.15.120:0/2675809679 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b6006fc60 con 0x7f4b600fdd20 2023-06-23T01:18:33.247 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.237+0000 7f4b5cff9700 1 -- 172.21.15.120:0/2675809679 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b5400f040 con 0x7f4b600fdd20 2023-06-23T01:18:33.247 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.237+0000 7f4b5cff9700 1 -- 172.21.15.120:0/2675809679 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5400c8c0 con 0x7f4b600fdd20 2023-06-23T01:18:33.248 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.237+0000 7f4b6579d700 1 -- 172.21.15.120:0/2675809679 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b60070220 con 0x7f4b600fdd20 2023-06-23T01:18:33.248 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.238+0000 7f4b6579d700 1 -- 172.21.15.120:0/2675809679 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b60103f90 con 0x7f4b600fdd20 2023-06-23T01:18:33.249 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.240+0000 7f4b5cff9700 1 -- 172.21.15.120:0/2675809679 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7f4b5400ccf0 con 0x7f4b600fdd20 2023-06-23T01:18:33.250 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.241+0000 7f4b5cff9700 1 --2- 172.21.15.120:0/2675809679 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f4b4809b680 0x7f4b4809db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:33.250 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.241+0000 7f4b5cff9700 1 -- 172.21.15.120:0/2675809679 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f4b540c44c0 con 0x7f4b600fdd20 2023-06-23T01:18:33.250 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.241+0000 7f4b5ffff700 1 --2- 172.21.15.120:0/2675809679 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f4b4809b680 0x7f4b4809db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:33.250 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.242+0000 7f4b5ffff700 1 --2- 172.21.15.120:0/2675809679 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f4b4809b680 0x7f4b4809db30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4b50004560 tx=0x7f4b50009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:33.251 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.245+0000 7f4b5cff9700 1 -- 172.21.15.120:0/2675809679 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f4b5404da00 con 0x7f4b600fdd20 2023-06-23T01:18:33.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:32 smithi120 ceph-mon[79813]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:33.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:32 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:33.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:32 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:33.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:32 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:33.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:32 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:33.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:32 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:33.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:32 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:18:33.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:32 smithi157 ceph-mon[116945]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:33.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:32 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:33.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:32 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:33.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:32 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:33.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:32 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:33.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:32 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:33.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:32 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:18:33.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:32 smithi134 ceph-mon[116146]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:33.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:32 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:33.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:32 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:33.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:32 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:33.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:32 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:33.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:32 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:33.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:32 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:18:33.504 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.501+0000 7f4b6579d700 1 -- 172.21.15.120:0/2675809679 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b60108300 con 0x7f4b600fdd20 2023-06-23T01:18:33.505 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.502+0000 7f4b5cff9700 1 -- 172.21.15.120:0/2675809679 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9625 (secure 0 0 0) 0x7f4b54090120 con 0x7f4b600fdd20 2023-06-23T01:18:33.505 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:18:33.506 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14427,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1538161586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1538161586},{"type":"v1","addr":"172.21.15.120:6835","nonce":1538161586}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/2588400819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":2588400819},{"type":"v1","addr":"172.21.15.120:6837","nonce":2588400819}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/1756726791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1756726791},{"type":"v1","addr":"172.21.15.120:6839","nonce":1756726791}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/2429971001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":2429971001},{"type":"v1","addr":"172.21.15.120:6841","nonce":2429971001}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3214085078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3214085078},{"type":"v1","addr":"172.21.15.134:6833","nonce":3214085078}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1478747127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1478747127},{"type":"v1","addr":"172.21.15.157:6833","nonce":1478747127}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/4238446318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":4238446318},{"type":"v1","addr":"172.21.15.134:6835","nonce":4238446318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2129674435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2129674435},{"type":"v1","addr":"172.21.15.157:6835","nonce":2129674435}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3499504142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3499504142},{"type":"v1","addr":"172.21.15.134:6837","nonce":3499504142}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1606323168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1606323168},{"type":"v1","addr":"172.21.15.157:6837","nonce":1606323168}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24559,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/4251766478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4251766478},{"type":"v1","addr":"172.21.15.157:6839","nonce":4251766478}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24581,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/1531909538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1531909538},{"type":"v1","addr":"172.21.15.134:6839","nonce":1531909538}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2023-06-23T01:18:33.508 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.505+0000 7f4b6579d700 1 -- 172.21.15.120:0/2675809679 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f4b4809b680 msgr2=0x7f4b4809db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:33.508 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.505+0000 7f4b6579d700 1 --2- 172.21.15.120:0/2675809679 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f4b4809b680 0x7f4b4809db30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4b50004560 tx=0x7f4b50009e30 comp rx=0 tx=0).stop 2023-06-23T01:18:33.508 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.505+0000 7f4b6579d700 1 -- 172.21.15.120:0/2675809679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b600fdd20 msgr2=0x7f4b6010dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:33.508 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.505+0000 7f4b6579d700 1 --2- 172.21.15.120:0/2675809679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b600fdd20 0x7f4b6010dc90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4b5400ebc0 tx=0x7f4b5400b040 comp rx=0 tx=0).stop 2023-06-23T01:18:33.508 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.505+0000 7f4b6579d700 1 -- 172.21.15.120:0/2675809679 shutdown_connections 2023-06-23T01:18:33.509 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.505+0000 7f4b6579d700 1 --2- 172.21.15.120:0/2675809679 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f4b4809b680 0x7f4b4809db30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:33.509 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.505+0000 7f4b6579d700 1 --2- 172.21.15.120:0/2675809679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b600fcb10 0x7f4b6010d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:33.509 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.505+0000 7f4b6579d700 1 --2- 172.21.15.120:0/2675809679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b600fe6b0 0x7f4b60114d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:33.509 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.505+0000 7f4b6579d700 1 --2- 172.21.15.120:0/2675809679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b600fdd20 0x7f4b6010dc90 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:33.509 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.505+0000 7f4b6579d700 1 -- 172.21.15.120:0/2675809679 >> 172.21.15.120:0/2675809679 conn(0x7f4b600f8230 msgr2=0x7f4b60101110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:33.509 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.505+0000 7f4b6579d700 1 -- 172.21.15.120:0/2675809679 shutdown_connections 2023-06-23T01:18:33.509 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.505+0000 7f4b6579d700 1 -- 172.21.15.120:0/2675809679 wait complete. 2023-06-23T01:18:33.510 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 13 2023-06-23T01:18:33.527 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T01:18:33.716 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.713+0000 7f84d7fff700 1 -- 172.21.15.120:0/2058774528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84d80fcb10 msgr2=0x7f84d80fcf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:33.716 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.713+0000 7f84d7fff700 1 --2- 172.21.15.120:0/2058774528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84d80fcb10 0x7f84d80fcf20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f84c80049c0 tx=0x7f84c8009e30 comp rx=0 tx=0).stop 2023-06-23T01:18:33.717 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.713+0000 7f84d7fff700 1 -- 172.21.15.120:0/2058774528 shutdown_connections 2023-06-23T01:18:33.717 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.713+0000 7f84d7fff700 1 --2- 172.21.15.120:0/2058774528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84d80fe6b0 0x7f84d8104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:33.717 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.713+0000 7f84d7fff700 1 --2- 172.21.15.120:0/2058774528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84d80fcb10 0x7f84d80fcf20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:33.717 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.713+0000 7f84d7fff700 1 --2- 172.21.15.120:0/2058774528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84d80fdd20 0x7f84d80fe170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:33.717 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.713+0000 7f84d7fff700 1 -- 172.21.15.120:0/2058774528 >> 172.21.15.120:0/2058774528 conn(0x7f84d80f8270 msgr2=0x7f84d80fa6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:33.717 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.713+0000 7f84d7fff700 1 -- 172.21.15.120:0/2058774528 shutdown_connections 2023-06-23T01:18:33.717 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.713+0000 7f84d7fff700 1 -- 172.21.15.120:0/2058774528 wait complete. 2023-06-23T01:18:33.719 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.714+0000 7f84d7fff700 1 Processor -- start 2023-06-23T01:18:33.719 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.714+0000 7f84d7fff700 1 -- start start 2023-06-23T01:18:33.719 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.714+0000 7f84d7fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84d80fcb10 0x7f84d8071720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:33.719 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.714+0000 7f84d7fff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84d80fdd20 0x7f84d806fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:33.720 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.714+0000 7f84d7fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84d80fe6b0 0x7f84d80702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:33.720 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.714+0000 7f84d7fff700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d8070a30 con 0x7f84d80fdd20 2023-06-23T01:18:33.720 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.714+0000 7f84d7fff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d8070ba0 con 0x7f84d80fcb10 2023-06-23T01:18:33.720 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.714+0000 7f84d7fff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d8117880 con 0x7f84d80fe6b0 2023-06-23T01:18:33.720 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.714+0000 7f84d6ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84d80fcb10 0x7f84d8071720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:33.720 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.714+0000 7f84d67fc700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84d80fdd20 0x7f84d806fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:33.720 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.714+0000 7f84d6ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84d80fcb10 0x7f84d8071720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:51290/0 (socket says 172.21.15.120:51290) 2023-06-23T01:18:33.721 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.714+0000 7f84d6ffd700 1 -- 172.21.15.120:0/2148158475 learned_addr learned my addr 172.21.15.120:0/2148158475 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:18:33.721 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.715+0000 7f84d77fe700 1 --2- 172.21.15.120:0/2148158475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84d80fe6b0 0x7f84d80702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:33.721 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.715+0000 7f84d6ffd700 1 -- 172.21.15.120:0/2148158475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84d80fe6b0 msgr2=0x7f84d80702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:33.721 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.715+0000 7f84d6ffd700 1 --2- 172.21.15.120:0/2148158475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84d80fe6b0 0x7f84d80702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:33.721 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.715+0000 7f84d6ffd700 1 -- 172.21.15.120:0/2148158475 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84d80fdd20 msgr2=0x7f84d806fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:33.721 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.715+0000 7f84d6ffd700 1 --2- 172.21.15.120:0/2148158475 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84d80fdd20 0x7f84d806fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:33.721 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.715+0000 7f84d6ffd700 1 -- 172.21.15.120:0/2148158475 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c8005040 con 0x7f84d80fcb10 2023-06-23T01:18:33.721 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.715+0000 7f84d6ffd700 1 --2- 172.21.15.120:0/2148158475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84d80fcb10 0x7f84d8071720 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f84c80137e0 tx=0x7f84c8009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:33.722 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.715+0000 7f84bf7fe700 1 -- 172.21.15.120:0/2148158475 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c800e050 con 0x7f84d80fcb10 2023-06-23T01:18:33.722 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.715+0000 7f84bf7fe700 1 -- 172.21.15.120:0/2148158475 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84c80097b0 con 0x7f84d80fcb10 2023-06-23T01:18:33.722 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.715+0000 7f84bf7fe700 1 -- 172.21.15.120:0/2148158475 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c801be40 con 0x7f84d80fcb10 2023-06-23T01:18:33.722 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.715+0000 7f84d7fff700 1 -- 172.21.15.120:0/2148158475 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84d8117ad0 con 0x7f84d80fcb10 2023-06-23T01:18:33.722 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.716+0000 7f84d7fff700 1 -- 172.21.15.120:0/2148158475 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84d8117f60 con 0x7f84d80fcb10 2023-06-23T01:18:33.723 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.716+0000 7f84d7fff700 1 -- 172.21.15.120:0/2148158475 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84d8108350 con 0x7f84d80fcb10 2023-06-23T01:18:33.723 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.718+0000 7f84bf7fe700 1 -- 172.21.15.120:0/2148158475 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7f84c801b4f0 con 0x7f84d80fcb10 2023-06-23T01:18:33.723 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.720+0000 7f84bf7fe700 1 --2- 172.21.15.120:0/2148158475 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f84c009b700 0x7f84c009dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:33.723 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.720+0000 7f84bf7fe700 1 -- 172.21.15.120:0/2148158475 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f84c80c62d0 con 0x7f84d80fcb10 2023-06-23T01:18:33.723 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.720+0000 7f84d67fc700 1 --2- 172.21.15.120:0/2148158475 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f84c009b700 0x7f84c009dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:33.724 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.721+0000 7f84d67fc700 1 --2- 172.21.15.120:0/2148158475 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f84c009b700 0x7f84c009dbb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f84cc009b80 tx=0x7f84cc00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:33.724 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.722+0000 7f84bf7fe700 1 -- 172.21.15.120:0/2148158475 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f84c8091f30 con 0x7f84d80fcb10 2023-06-23T01:18:33.938 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.935+0000 7f84d7fff700 1 -- 172.21.15.120:0/2148158475 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84d81180f0 con 0x7f84d80fcb10 2023-06-23T01:18:33.939 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.936+0000 7f84bf7fe700 1 -- 172.21.15.120:0/2148158475 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18241 (secure 0 0 0) 0x7f84c8095550 con 0x7f84d80fcb10 2023-06-23T01:18:33.940 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:18:33.940 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":81,"fsid":"9efbfa8e-1162-11ee-9b2b-001a4aab830c","created":"2023-06-23T01:10:46.840474+0000","modified":"2023-06-23T01:17:47.828105+0000","last_up_change":"2023-06-23T01:17:46.670887+0000","last_in_change":"2023-06-23T01:17:27.552330+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T01:14:03.589464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"3729642b-e74f-4253-bde8-535d8bfe8c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3479894866},{"type":"v1","addr":"172.21.15.120:6803","nonce":3479894866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3479894866},{"type":"v1","addr":"172.21.15.120:6805","nonce":3479894866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3479894866},{"type":"v1","addr":"172.21.15.120:6809","nonce":3479894866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3479894866},{"type":"v1","addr":"172.21.15.120:6807","nonce":3479894866}]},"public_addr":"172.21.15.120:6803/3479894866","cluster_addr":"172.21.15.120:6805/3479894866","heartbeat_back_addr":"172.21.15.120:6809/3479894866","heartbeat_front_addr":"172.21.15.120:6807/3479894866","state":["exists","up"]},{"osd":1,"uuid":"58240258-afd5-4991-9d5b-a80d365e4fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3046662876},{"type":"v1","addr":"172.21.15.134:6801","nonce":3046662876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3046662876},{"type":"v1","addr":"172.21.15.134:6803","nonce":3046662876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3046662876},{"type":"v1","addr":"172.21.15.134:6807","nonce":3046662876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3046662876},{"type":"v1","addr":"172.21.15.134:6805","nonce":3046662876}]},"public_addr":"172.21.15.134:6801/3046662876","cluster_addr":"172.21.15.134:6803/3046662876","heartbeat_back_addr":"172.21.15.134:6807/3046662876","heartbeat_front_addr":"172.21.15.134:6805/3046662876","state":["exists","up"]},{"osd":2,"uuid":"05988a7a-7cf2-4fbb-9d4f-22604cd2f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3996265223},{"type":"v1","addr":"172.21.15.157:6801","nonce":3996265223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3996265223},{"type":"v1","addr":"172.21.15.157:6803","nonce":3996265223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3996265223},{"type":"v1","addr":"172.21.15.157:6807","nonce":3996265223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3996265223},{"type":"v1","addr":"172.21.15.157:6805","nonce":3996265223}]},"public_addr":"172.21.15.157:6801/3996265223","cluster_addr":"172.21.15.157:6803/3996265223","heartbeat_back_addr":"172.21.15.157:6807/3996265223","heartbeat_front_addr":"172.21.15.157:6805/3996265223","state":["exists","up"]},{"osd":3,"uuid":"7d9e9739-f043-43d6-8e46-7711dcf43e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":1716638437},{"type":"v1","addr":"172.21.15.120:6811","nonce":1716638437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":1716638437},{"type":"v1","addr":"172.21.15.120:6813","nonce":1716638437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1716638437},{"type":"v1","addr":"172.21.15.120:6817","nonce":1716638437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":1716638437},{"type":"v1","addr":"172.21.15.120:6815","nonce":1716638437}]},"public_addr":"172.21.15.120:6811/1716638437","cluster_addr":"172.21.15.120:6813/1716638437","heartbeat_back_addr":"172.21.15.120:6817/1716638437","heartbeat_front_addr":"172.21.15.120:6815/1716638437","state":["exists","up"]},{"osd":4,"uuid":"3a70f86d-f903-4618-be66-9aa1eac44ccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4172238262},{"type":"v1","addr":"172.21.15.134:6809","nonce":4172238262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4172238262},{"type":"v1","addr":"172.21.15.134:6811","nonce":4172238262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4172238262},{"type":"v1","addr":"172.21.15.134:6815","nonce":4172238262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4172238262},{"type":"v1","addr":"172.21.15.134:6813","nonce":4172238262}]},"public_addr":"172.21.15.134:6809/4172238262","cluster_addr":"172.21.15.134:6811/4172238262","heartbeat_back_addr":"172.21.15.134:6815/4172238262","heartbeat_front_addr":"172.21.15.134:6813/4172238262","state":["exists","up"]},{"osd":5,"uuid":"b2316488-75c3-4847-b4c4-998730f27c88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3374822045},{"type":"v1","addr":"172.21.15.157:6809","nonce":3374822045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3374822045},{"type":"v1","addr":"172.21.15.157:6811","nonce":3374822045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3374822045},{"type":"v1","addr":"172.21.15.157:6815","nonce":3374822045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3374822045},{"type":"v1","addr":"172.21.15.157:6813","nonce":3374822045}]},"public_addr":"172.21.15.157:6809/3374822045","cluster_addr":"172.21.15.157:6811/3374822045","heartbeat_back_addr":"172.21.15.157:6815/3374822045","heartbeat_front_addr":"172.21.15.157:6813/3374822045","state":["exists","up"]},{"osd":6,"uuid":"338c2c4a-d982-4031-aaaf-be76c9dd74a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2097345225},{"type":"v1","addr":"172.21.15.120:6819","nonce":2097345225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2097345225},{"type":"v1","addr":"172.21.15.120:6821","nonce":2097345225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2097345225},{"type":"v1","addr":"172.21.15.120:6825","nonce":2097345225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2097345225},{"type":"v1","addr":"172.21.15.120:6823","nonce":2097345225}]},"public_addr":"172.21.15.120:6819/2097345225","cluster_addr":"172.21.15.120:6821/2097345225","heartbeat_back_addr":"172.21.15.120:6825/2097345225","heartbeat_front_addr":"172.21.15.120:6823/2097345225","state":["exists","up"]},{"osd":7,"uuid":"bff92799-ae9c-4409-be65-7ffc27a1c3f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1345745298},{"type":"v1","addr":"172.21.15.134:6817","nonce":1345745298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1345745298},{"type":"v1","addr":"172.21.15.134:6819","nonce":1345745298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1345745298},{"type":"v1","addr":"172.21.15.134:6823","nonce":1345745298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1345745298},{"type":"v1","addr":"172.21.15.134:6821","nonce":1345745298}]},"public_addr":"172.21.15.134:6817/1345745298","cluster_addr":"172.21.15.134:6819/1345745298","heartbeat_back_addr":"172.21.15.134:6823/1345745298","heartbeat_front_addr":"172.21.15.134:6821/1345745298","state":["exists","up"]},{"osd":8,"uuid":"e4aac47e-d7f9-4feb-ada7-d8103fdca246","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1124549599},{"type":"v1","addr":"172.21.15.157:6817","nonce":1124549599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1124549599},{"type":"v1","addr":"172.21.15.157:6819","nonce":1124549599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1124549599},{"type":"v1","addr":"172.21.15.157:6823","nonce":1124549599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1124549599},{"type":"v1","addr":"172.21.15.157:6821","nonce":1124549599}]},"public_addr":"172.21.15.157:6817/1124549599","cluster_addr":"172.21.15.157:6819/1124549599","heartbeat_back_addr":"172.21.15.157:6823/1124549599","heartbeat_front_addr":"172.21.15.157:6821/1124549599","state":["exists","up"]},{"osd":9,"uuid":"48b988ba-e866-47d1-aca9-09f6ad815bd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1402414981},{"type":"v1","addr":"172.21.15.120:6827","nonce":1402414981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1402414981},{"type":"v1","addr":"172.21.15.120:6829","nonce":1402414981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1402414981},{"type":"v1","addr":"172.21.15.120:6833","nonce":1402414981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1402414981},{"type":"v1","addr":"172.21.15.120:6831","nonce":1402414981}]},"public_addr":"172.21.15.120:6827/1402414981","cluster_addr":"172.21.15.120:6829/1402414981","heartbeat_back_addr":"172.21.15.120:6833/1402414981","heartbeat_front_addr":"172.21.15.120:6831/1402414981","state":["exists","up"]},{"osd":10,"uuid":"777f3add-3779-4221-a86f-65cca2e5e8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":4041441144},{"type":"v1","addr":"172.21.15.134:6825","nonce":4041441144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":4041441144},{"type":"v1","addr":"172.21.15.134:6827","nonce":4041441144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":4041441144},{"type":"v1","addr":"172.21.15.134:6831","nonce":4041441144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":4041441144},{"type":"v1","addr":"172.21.15.134:6829","nonce":4041441144}]},"public_addr":"172.21.15.134:6825/4041441144","cluster_addr":"172.21.15.134:6827/4041441144","heartbeat_back_addr":"172.21.15.134:6831/4041441144","heartbeat_front_addr":"172.21.15.134:6829/4041441144","state":["exists","up"]},{"osd":11,"uuid":"7166e11b-03c0-45bf-abbf-a74c4a13a39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1537477260},{"type":"v1","addr":"172.21.15.157:6825","nonce":1537477260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1537477260},{"type":"v1","addr":"172.21.15.157:6827","nonce":1537477260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1537477260},{"type":"v1","addr":"172.21.15.157:6831","nonce":1537477260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1537477260},{"type":"v1","addr":"172.21.15.157:6829","nonce":1537477260}]},"public_addr":"172.21.15.157:6825/1537477260","cluster_addr":"172.21.15.157:6827/1537477260","heartbeat_back_addr":"172.21.15.157:6831/1537477260","heartbeat_front_addr":"172.21.15.157:6829/1537477260","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:13:13.580631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:13:37.472924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:14:01.591555+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:14:28.472130+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:14:52.356433+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:15:15.946523+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:15:42.036105+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:16:06.154934+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:16:30.043788+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:16:55.975403+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:17:20.612778+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:17:44.896301+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.120:0/2865552846":"2023-06-24T01:11:38.967204+0000","172.21.15.120:0/2163781512":"2023-06-24T01:11:38.967204+0000","172.21.15.120:0/1943345772":"2023-06-24T01:11:38.967204+0000","172.21.15.120:6801/1836713308":"2023-06-24T01:11:38.967204+0000","172.21.15.120:0/2358086231":"2023-06-24T01:11:14.574788+0000","172.21.15.120:6801/506843621":"2023-06-24T01:11:14.574788+0000","172.21.15.120:6800/506843621":"2023-06-24T01:11:14.574788+0000","172.21.15.120:0/4077038498":"2023-06-24T01:11:14.574788+0000","172.21.15.120:6800/1836713308":"2023-06-24T01:11:38.967204+0000","172.21.15.120:0/2484415475":"2023-06-24T01:11:14.574788+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T01:18:33.942 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.939+0000 7f84d7fff700 1 -- 172.21.15.120:0/2148158475 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f84c009b700 msgr2=0x7f84c009dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:33.942 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.939+0000 7f84d7fff700 1 --2- 172.21.15.120:0/2148158475 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f84c009b700 0x7f84c009dbb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f84cc009b80 tx=0x7f84cc00b040 comp rx=0 tx=0).stop 2023-06-23T01:18:33.943 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.939+0000 7f84d7fff700 1 -- 172.21.15.120:0/2148158475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84d80fcb10 msgr2=0x7f84d8071720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:33.943 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.939+0000 7f84d7fff700 1 --2- 172.21.15.120:0/2148158475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84d80fcb10 0x7f84d8071720 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f84c80137e0 tx=0x7f84c8009e30 comp rx=0 tx=0).stop 2023-06-23T01:18:33.943 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.939+0000 7f84d7fff700 1 -- 172.21.15.120:0/2148158475 shutdown_connections 2023-06-23T01:18:33.943 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.939+0000 7f84d7fff700 1 --2- 172.21.15.120:0/2148158475 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f84c009b700 0x7f84c009dbb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:33.943 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.939+0000 7f84d7fff700 1 --2- 172.21.15.120:0/2148158475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84d80fe6b0 0x7f84d80702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:33.943 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.939+0000 7f84d7fff700 1 --2- 172.21.15.120:0/2148158475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84d80fcb10 0x7f84d8071720 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:33.944 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.939+0000 7f84d7fff700 1 --2- 172.21.15.120:0/2148158475 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84d80fdd20 0x7f84d806fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:33.944 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.939+0000 7f84d7fff700 1 -- 172.21.15.120:0/2148158475 >> 172.21.15.120:0/2148158475 conn(0x7f84d80f8270 msgr2=0x7f84d80fa400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:33.944 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.939+0000 7f84d7fff700 1 -- 172.21.15.120:0/2148158475 shutdown_connections 2023-06-23T01:18:33.944 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:33.939+0000 7f84d7fff700 1 -- 172.21.15.120:0/2148158475 wait complete. 2023-06-23T01:18:33.962 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-23T01:14:03.589464+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-06-23T01:18:33.963 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T01:18:34.178 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.174+0000 7f3ef23ff700 1 -- 172.21.15.120:0/999839222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3eec102990 msgr2=0x7f3eec104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:34.178 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.174+0000 7f3ef23ff700 1 --2- 172.21.15.120:0/999839222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3eec102990 0x7f3eec104e40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009d60 tx=0x7f3ee4004d10 comp rx=0 tx=0).stop 2023-06-23T01:18:34.178 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.174+0000 7f3ef23ff700 1 -- 172.21.15.120:0/999839222 shutdown_connections 2023-06-23T01:18:34.178 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.174+0000 7f3ef23ff700 1 --2- 172.21.15.120:0/999839222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3eec102990 0x7f3eec104e40 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:34.179 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.174+0000 7f3ef23ff700 1 --2- 172.21.15.120:0/999839222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec065cb0 0x7f3eec0ff8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:34.179 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.174+0000 7f3ef23ff700 1 --2- 172.21.15.120:0/999839222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3eec0ffea0 0x7f3eec102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:34.179 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.174+0000 7f3ef23ff700 1 -- 172.21.15.120:0/999839222 >> 172.21.15.120:0/999839222 conn(0x7f3eec0f7380 msgr2=0x7f3eec0f97b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:34.179 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.174+0000 7f3ef23ff700 1 -- 172.21.15.120:0/999839222 shutdown_connections 2023-06-23T01:18:34.179 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.174+0000 7f3ef23ff700 1 -- 172.21.15.120:0/999839222 wait complete. 2023-06-23T01:18:34.181 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.175+0000 7f3ef23ff700 1 Processor -- start 2023-06-23T01:18:34.181 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.176+0000 7f3ef23ff700 1 -- start start 2023-06-23T01:18:34.182 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.176+0000 7f3ef23ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec065cb0 0x7f3eec0fd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:34.182 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.176+0000 7f3ef23ff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3eec0ffea0 0x7f3eec0fb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:34.182 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.176+0000 7f3ef23ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3eec102990 0x7f3eec0fbd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:34.182 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.176+0000 7f3ef23ff700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec0fc480 con 0x7f3eec0ffea0 2023-06-23T01:18:34.182 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.176+0000 7f3ef23ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec0fc5f0 con 0x7f3eec102990 2023-06-23T01:18:34.182 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.176+0000 7f3ef23ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec117810 con 0x7f3eec065cb0 2023-06-23T01:18:34.182 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.176+0000 7f3eeb7fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3eec0ffea0 0x7f3eec0fb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:34.183 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.176+0000 7f3eeb7fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3eec0ffea0 0x7f3eec0fb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38622/0 (socket says 172.21.15.120:38622) 2023-06-23T01:18:34.183 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.176+0000 7f3eeb7fe700 1 -- 172.21.15.120:0/1941468764 learned_addr learned my addr 172.21.15.120:0/1941468764 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:18:34.183 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.176+0000 7f3eebfff700 1 --2- 172.21.15.120:0/1941468764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec065cb0 0x7f3eec0fd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:34.183 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.176+0000 7f3ef099c700 1 --2- 172.21.15.120:0/1941468764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3eec102990 0x7f3eec0fbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:34.183 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.177+0000 7f3ef099c700 1 -- 172.21.15.120:0/1941468764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec065cb0 msgr2=0x7f3eec0fd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:34.183 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.177+0000 7f3ef099c700 1 --2- 172.21.15.120:0/1941468764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec065cb0 0x7f3eec0fd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:34.183 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.177+0000 7f3ef099c700 1 -- 172.21.15.120:0/1941468764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3eec0ffea0 msgr2=0x7f3eec0fb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:34.184 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.177+0000 7f3ef099c700 1 --2- 172.21.15.120:0/1941468764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3eec0ffea0 0x7f3eec0fb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:34.184 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.177+0000 7f3ef099c700 1 -- 172.21.15.120:0/1941468764 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee4009940 con 0x7f3eec102990 2023-06-23T01:18:34.184 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.177+0000 7f3eebfff700 1 --2- 172.21.15.120:0/1941468764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec065cb0 0x7f3eec0fd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T01:18:34.184 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.177+0000 7f3ef099c700 1 --2- 172.21.15.120:0/1941468764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3eec102990 0x7f3eec0fbd10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009d60 tx=0x7f3ee4004d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:34.184 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.177+0000 7f3ee8ff9700 1 -- 172.21.15.120:0/1941468764 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee400e030 con 0x7f3eec102990 2023-06-23T01:18:34.184 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.177+0000 7f3ee8ff9700 1 -- 172.21.15.120:0/1941468764 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ee4008ec0 con 0x7f3eec102990 2023-06-23T01:18:34.184 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.177+0000 7f3ef23ff700 1 -- 172.21.15.120:0/1941468764 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eec117a60 con 0x7f3eec102990 2023-06-23T01:18:34.185 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.177+0000 7f3ee8ff9700 1 -- 172.21.15.120:0/1941468764 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4012680 con 0x7f3eec102990 2023-06-23T01:18:34.185 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.178+0000 7f3ef23ff700 1 -- 172.21.15.120:0/1941468764 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eec117f90 con 0x7f3eec102990 2023-06-23T01:18:34.185 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.179+0000 7f3ef23ff700 1 -- 172.21.15.120:0/1941468764 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eec0f8eb0 con 0x7f3eec102990 2023-06-23T01:18:34.186 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.181+0000 7f3ee8ff9700 1 -- 172.21.15.120:0/1941468764 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7f3ee4012ab0 con 0x7f3eec102990 2023-06-23T01:18:34.186 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.183+0000 7f3ee8ff9700 1 --2- 172.21.15.120:0/1941468764 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f3ed409b7b0 0x7f3ed409dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:34.187 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.183+0000 7f3ee8ff9700 1 -- 172.21.15.120:0/1941468764 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f3ee40c6ff0 con 0x7f3eec102990 2023-06-23T01:18:34.187 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.183+0000 7f3eebfff700 1 --2- 172.21.15.120:0/1941468764 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f3ed409b7b0 0x7f3ed409dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:34.187 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.184+0000 7f3eebfff700 1 --2- 172.21.15.120:0/1941468764 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f3ed409b7b0 0x7f3ed409dc60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3edc004600 tx=0x7f3edc00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:34.187 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.184+0000 7f3ee8ff9700 1 -- 172.21.15.120:0/1941468764 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f3ee4092c70 con 0x7f3eec102990 2023-06-23T01:18:34.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:34 smithi120 ceph-mon[79813]: mds.? [v2:172.21.15.134:6838/1531909538,v1:172.21.15.134:6839/1531909538] up:boot 2023-06-23T01:18:34.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:34 smithi120 ceph-mon[79813]: fsmap 12 up:standby 2023-06-23T01:18:34.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:34 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-06-23T01:18:34.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:34 smithi120 ceph-mon[79813]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:34.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:34 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/2675809679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:34.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:34 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/2148158475' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T01:18:34.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:34 smithi157 ceph-mon[116945]: mds.? [v2:172.21.15.134:6838/1531909538,v1:172.21.15.134:6839/1531909538] up:boot 2023-06-23T01:18:34.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:34 smithi157 ceph-mon[116945]: fsmap 12 up:standby 2023-06-23T01:18:34.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:34 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-06-23T01:18:34.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:34 smithi157 ceph-mon[116945]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:34.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:34 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/2675809679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:34.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:34 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/2148158475' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T01:18:34.397 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.394+0000 7f3ef23ff700 1 -- 172.21.15.120:0/1941468764 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3eec108710 con 0x7f3eec102990 2023-06-23T01:18:34.397 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.395+0000 7f3ee8ff9700 1 -- 172.21.15.120:0/1941468764 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3eec108710 con 0x7f3eec102990 2023-06-23T01:18:34.398 INFO:teuthology.orchestra.run.smithi120.stdout:pg_num: 1 2023-06-23T01:18:34.401 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.397+0000 7f3ef23ff700 1 -- 172.21.15.120:0/1941468764 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f3ed409b7b0 msgr2=0x7f3ed409dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:34.401 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.397+0000 7f3ef23ff700 1 --2- 172.21.15.120:0/1941468764 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f3ed409b7b0 0x7f3ed409dc60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3edc004600 tx=0x7f3edc00b040 comp rx=0 tx=0).stop 2023-06-23T01:18:34.401 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.397+0000 7f3ef23ff700 1 -- 172.21.15.120:0/1941468764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3eec102990 msgr2=0x7f3eec0fbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:34.402 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.397+0000 7f3ef23ff700 1 --2- 172.21.15.120:0/1941468764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3eec102990 0x7f3eec0fbd10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009d60 tx=0x7f3ee4004d10 comp rx=0 tx=0).stop 2023-06-23T01:18:34.402 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.397+0000 7f3ef23ff700 1 -- 172.21.15.120:0/1941468764 shutdown_connections 2023-06-23T01:18:34.402 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.397+0000 7f3ef23ff700 1 --2- 172.21.15.120:0/1941468764 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f3ed409b7b0 0x7f3ed409dc60 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:34.403 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.397+0000 7f3ef23ff700 1 --2- 172.21.15.120:0/1941468764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3eec102990 0x7f3eec0fbd10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:34.403 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.397+0000 7f3ef23ff700 1 --2- 172.21.15.120:0/1941468764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec065cb0 0x7f3eec0fd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:34.404 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.397+0000 7f3ef23ff700 1 --2- 172.21.15.120:0/1941468764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3eec0ffea0 0x7f3eec0fb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:34.404 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.397+0000 7f3ef23ff700 1 -- 172.21.15.120:0/1941468764 >> 172.21.15.120:0/1941468764 conn(0x7f3eec0f7380 msgr2=0x7f3eec0fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:34.404 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.398+0000 7f3ef23ff700 1 -- 172.21.15.120:0/1941468764 shutdown_connections 2023-06-23T01:18:34.404 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.398+0000 7f3ef23ff700 1 -- 172.21.15.120:0/1941468764 wait complete. 2023-06-23T01:18:34.418 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-06-23T01:18:34.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:34 smithi134 ceph-mon[116146]: mds.? [v2:172.21.15.134:6838/1531909538,v1:172.21.15.134:6839/1531909538] up:boot 2023-06-23T01:18:34.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:34 smithi134 ceph-mon[116146]: fsmap 12 up:standby 2023-06-23T01:18:34.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:34 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-06-23T01:18:34.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:34 smithi134 ceph-mon[116146]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:34.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:34 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/2675809679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T01:18:34.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:34 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/2148158475' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T01:18:34.610 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.607+0000 7fde51b9d700 1 -- 172.21.15.120:0/1007383976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde4c066bb0 msgr2=0x7fde4c0ff6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:34.610 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.607+0000 7fde51b9d700 1 --2- 172.21.15.120:0/1007383976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde4c066bb0 0x7fde4c0ff6c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fde40009d60 tx=0x7fde40004d10 comp rx=0 tx=0).stop 2023-06-23T01:18:34.610 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.607+0000 7fde51b9d700 1 -- 172.21.15.120:0/1007383976 shutdown_connections 2023-06-23T01:18:34.610 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.607+0000 7fde51b9d700 1 --2- 172.21.15.120:0/1007383976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde4c066bb0 0x7fde4c0ff6c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:34.610 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.607+0000 7fde51b9d700 1 --2- 172.21.15.120:0/1007383976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde4c102750 0x7fde4c104c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:34.611 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.607+0000 7fde51b9d700 1 --2- 172.21.15.120:0/1007383976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde4c0ffc00 0x7fde4c102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:34.611 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.607+0000 7fde51b9d700 1 -- 172.21.15.120:0/1007383976 >> 172.21.15.120:0/1007383976 conn(0x7fde4c0f7130 msgr2=0x7fde4c0f9560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:34.611 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.607+0000 7fde51b9d700 1 -- 172.21.15.120:0/1007383976 shutdown_connections 2023-06-23T01:18:34.611 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.607+0000 7fde51b9d700 1 -- 172.21.15.120:0/1007383976 wait complete. 2023-06-23T01:18:34.613 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.607+0000 7fde51b9d700 1 Processor -- start 2023-06-23T01:18:34.613 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.608+0000 7fde51b9d700 1 -- start start 2023-06-23T01:18:34.613 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.608+0000 7fde51b9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde4c066bb0 0x7fde4c10d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:34.613 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.608+0000 7fde51b9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde4c0ffc00 0x7fde4c10dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:34.613 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.608+0000 7fde51b9d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde4c102750 0x7fde4c114e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:34.614 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.608+0000 7fde51b9d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c115430 con 0x7fde4c102750 2023-06-23T01:18:34.614 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.608+0000 7fde51b9d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c1155a0 con 0x7fde4c0ffc00 2023-06-23T01:18:34.614 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.608+0000 7fde51b9d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c115760 con 0x7fde4c066bb0 2023-06-23T01:18:34.614 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.608+0000 7fde4bfff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde4c0ffc00 0x7fde4c10dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:34.614 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.608+0000 7fde5139c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde4c102750 0x7fde4c114e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:34.614 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.608+0000 7fde4bfff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde4c0ffc00 0x7fde4c10dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:51336/0 (socket says 172.21.15.120:51336) 2023-06-23T01:18:34.614 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.608+0000 7fde4bfff700 1 -- 172.21.15.120:0/533398391 learned_addr learned my addr 172.21.15.120:0/533398391 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:18:34.615 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.608+0000 7fde50b9b700 1 --2- 172.21.15.120:0/533398391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde4c066bb0 0x7fde4c10d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:34.615 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.608+0000 7fde4bfff700 1 -- 172.21.15.120:0/533398391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde4c066bb0 msgr2=0x7fde4c10d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:34.615 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.609+0000 7fde4bfff700 1 --2- 172.21.15.120:0/533398391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde4c066bb0 0x7fde4c10d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:34.615 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.609+0000 7fde4bfff700 1 -- 172.21.15.120:0/533398391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde4c102750 msgr2=0x7fde4c114e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:34.615 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.609+0000 7fde4bfff700 1 --2- 172.21.15.120:0/533398391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde4c102750 0x7fde4c114e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:34.615 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.609+0000 7fde4bfff700 1 -- 172.21.15.120:0/533398391 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde40009940 con 0x7fde4c0ffc00 2023-06-23T01:18:34.615 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.609+0000 7fde5139c700 1 --2- 172.21.15.120:0/533398391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde4c102750 0x7fde4c114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T01:18:34.615 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.609+0000 7fde4bfff700 1 --2- 172.21.15.120:0/533398391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde4c0ffc00 0x7fde4c10dd90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fde38002b00 tx=0x7fde380046b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:34.616 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.609+0000 7fde497fa700 1 -- 172.21.15.120:0/533398391 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde38007cc0 con 0x7fde4c0ffc00 2023-06-23T01:18:34.616 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.609+0000 7fde51b9d700 1 -- 172.21.15.120:0/533398391 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde4c0fb500 con 0x7fde4c0ffc00 2023-06-23T01:18:34.616 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.609+0000 7fde497fa700 1 -- 172.21.15.120:0/533398391 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde3800f040 con 0x7fde4c0ffc00 2023-06-23T01:18:34.616 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.609+0000 7fde497fa700 1 -- 172.21.15.120:0/533398391 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3800d900 con 0x7fde4c0ffc00 2023-06-23T01:18:34.616 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.609+0000 7fde51b9d700 1 -- 172.21.15.120:0/533398391 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde4c0fb9f0 con 0x7fde4c0ffc00 2023-06-23T01:18:34.616 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.610+0000 7fde51b9d700 1 -- 172.21.15.120:0/533398391 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde4c0f8c60 con 0x7fde4c0ffc00 2023-06-23T01:18:34.617 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.612+0000 7fde497fa700 1 -- 172.21.15.120:0/533398391 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7fde3800dd30 con 0x7fde4c0ffc00 2023-06-23T01:18:34.617 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.614+0000 7fde497fa700 1 --2- 172.21.15.120:0/533398391 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fde3c09b680 0x7fde3c09db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:34.617 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.614+0000 7fde497fa700 1 -- 172.21.15.120:0/533398391 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7fde380c4850 con 0x7fde4c0ffc00 2023-06-23T01:18:34.618 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.614+0000 7fde50b9b700 1 --2- 172.21.15.120:0/533398391 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fde3c09b680 0x7fde3c09db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:34.618 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.614+0000 7fde50b9b700 1 --2- 172.21.15.120:0/533398391 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fde3c09b680 0x7fde3c09db30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fde40015040 tx=0x7fde40004d10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:34.618 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.615+0000 7fde497fa700 1 -- 172.21.15.120:0/533398391 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fde380904b0 con 0x7fde4c0ffc00 2023-06-23T01:18:34.834 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.831+0000 7fde51b9d700 1 -- 172.21.15.120:0/533398391 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fde4c02b210 con 0x7fde4c0ffc00 2023-06-23T01:18:34.834 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.832+0000 7fde497fa700 1 -- 172.21.15.120:0/533398391 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) v1 ==== 84+0+65 (secure 0 0 0) 0x7fde38093ad0 con 0x7fde4c0ffc00 2023-06-23T01:18:34.835 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:18:34.835 INFO:teuthology.orchestra.run.smithi120.stdout:[ 2023-06-23T01:18:34.835 INFO:teuthology.orchestra.run.smithi120.stdout: { 2023-06-23T01:18:34.835 INFO:teuthology.orchestra.run.smithi120.stdout: "poolnum": 1, 2023-06-23T01:18:34.835 INFO:teuthology.orchestra.run.smithi120.stdout: "poolname": ".mgr" 2023-06-23T01:18:34.836 INFO:teuthology.orchestra.run.smithi120.stdout: } 2023-06-23T01:18:34.836 INFO:teuthology.orchestra.run.smithi120.stdout:] 2023-06-23T01:18:34.837 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.834+0000 7fde51b9d700 1 -- 172.21.15.120:0/533398391 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fde3c09b680 msgr2=0x7fde3c09db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:34.837 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.834+0000 7fde51b9d700 1 --2- 172.21.15.120:0/533398391 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fde3c09b680 0x7fde3c09db30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fde40015040 tx=0x7fde40004d10 comp rx=0 tx=0).stop 2023-06-23T01:18:34.838 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.834+0000 7fde51b9d700 1 -- 172.21.15.120:0/533398391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde4c0ffc00 msgr2=0x7fde4c10dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:34.838 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.834+0000 7fde51b9d700 1 --2- 172.21.15.120:0/533398391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde4c0ffc00 0x7fde4c10dd90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fde38002b00 tx=0x7fde380046b0 comp rx=0 tx=0).stop 2023-06-23T01:18:34.838 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.834+0000 7fde51b9d700 1 -- 172.21.15.120:0/533398391 shutdown_connections 2023-06-23T01:18:34.838 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.834+0000 7fde51b9d700 1 --2- 172.21.15.120:0/533398391 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fde3c09b680 0x7fde3c09db30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:34.838 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.834+0000 7fde51b9d700 1 --2- 172.21.15.120:0/533398391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde4c102750 0x7fde4c114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:34.838 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.834+0000 7fde51b9d700 1 --2- 172.21.15.120:0/533398391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde4c066bb0 0x7fde4c10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:34.838 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.834+0000 7fde51b9d700 1 --2- 172.21.15.120:0/533398391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde4c0ffc00 0x7fde4c10dd90 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:34.839 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.834+0000 7fde51b9d700 1 -- 172.21.15.120:0/533398391 >> 172.21.15.120:0/533398391 conn(0x7fde4c0f7130 msgr2=0x7fde4c0fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:34.839 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.834+0000 7fde51b9d700 1 -- 172.21.15.120:0/533398391 shutdown_connections 2023-06-23T01:18:34.839 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:34.834+0000 7fde51b9d700 1 -- 172.21.15.120:0/533398391 wait complete. 2023-06-23T01:18:34.855 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-06-23T01:18:34.856 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-06-23T01:18:35.059 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.056+0000 7fda4599d700 1 -- 172.21.15.120:0/3056693017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda40065c10 msgr2=0x7fda400ff710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:35.060 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.056+0000 7fda4599d700 1 --2- 172.21.15.120:0/3056693017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda40065c10 0x7fda400ff710 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fda340049c0 tx=0x7fda34009e30 comp rx=0 tx=0).stop 2023-06-23T01:18:35.060 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.056+0000 7fda4599d700 1 -- 172.21.15.120:0/3056693017 shutdown_connections 2023-06-23T01:18:35.060 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.056+0000 7fda4599d700 1 --2- 172.21.15.120:0/3056693017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda40102740 0x7fda40104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:35.060 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.056+0000 7fda4599d700 1 --2- 172.21.15.120:0/3056693017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda40065c10 0x7fda400ff710 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:35.060 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.056+0000 7fda4599d700 1 --2- 172.21.15.120:0/3056693017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda400ffc50 0x7fda40102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:35.061 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.056+0000 7fda4599d700 1 -- 172.21.15.120:0/3056693017 >> 172.21.15.120:0/3056693017 conn(0x7fda400f7100 msgr2=0x7fda400f9550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:35.061 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.056+0000 7fda4599d700 1 -- 172.21.15.120:0/3056693017 shutdown_connections 2023-06-23T01:18:35.061 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.056+0000 7fda4599d700 1 -- 172.21.15.120:0/3056693017 wait complete. 2023-06-23T01:18:35.064 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.057+0000 7fda4599d700 1 Processor -- start 2023-06-23T01:18:35.065 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.057+0000 7fda4599d700 1 -- start start 2023-06-23T01:18:35.065 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.057+0000 7fda4599d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda40065c10 0x7fda4010fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:35.065 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.057+0000 7fda4599d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda400ffc50 0x7fda4010ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:35.065 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.057+0000 7fda4599d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda40102740 0x7fda40117010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:35.066 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.057+0000 7fda4599d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda40117620 con 0x7fda400ffc50 2023-06-23T01:18:35.066 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.057+0000 7fda4599d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda40117790 con 0x7fda40102740 2023-06-23T01:18:35.066 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.057+0000 7fda4599d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda40117950 con 0x7fda40065c10 2023-06-23T01:18:35.067 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.058+0000 7fda4499b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda40065c10 0x7fda4010fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:35.067 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.058+0000 7fda4499b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda40065c10 0x7fda4010fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.120:49414/0 (socket says 172.21.15.120:49414) 2023-06-23T01:18:35.067 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.058+0000 7fda3ffff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda400ffc50 0x7fda4010ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:35.067 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.058+0000 7fda4499b700 1 -- 172.21.15.120:0/3495058307 learned_addr learned my addr 172.21.15.120:0/3495058307 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:18:35.068 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.058+0000 7fda4519c700 1 --2- 172.21.15.120:0/3495058307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda40102740 0x7fda40117010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:35.068 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.058+0000 7fda4499b700 1 -- 172.21.15.120:0/3495058307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda40102740 msgr2=0x7fda40117010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:35.068 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.058+0000 7fda4499b700 1 --2- 172.21.15.120:0/3495058307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda40102740 0x7fda40117010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:35.068 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.058+0000 7fda4499b700 1 -- 172.21.15.120:0/3495058307 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda400ffc50 msgr2=0x7fda4010ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:35.069 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.058+0000 7fda4499b700 1 --2- 172.21.15.120:0/3495058307 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda400ffc50 0x7fda4010ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:35.069 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.058+0000 7fda4499b700 1 -- 172.21.15.120:0/3495058307 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda34005040 con 0x7fda40065c10 2023-06-23T01:18:35.069 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.058+0000 7fda4499b700 1 --2- 172.21.15.120:0/3495058307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda40065c10 0x7fda4010fa40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fda34009150 tx=0x7fda34009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:35.069 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.058+0000 7fda3d7fa700 1 -- 172.21.15.120:0/3495058307 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda34020070 con 0x7fda40065c10 2023-06-23T01:18:35.070 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.058+0000 7fda3d7fa700 1 -- 172.21.15.120:0/3495058307 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda340041a0 con 0x7fda40065c10 2023-06-23T01:18:35.070 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.058+0000 7fda4599d700 1 -- 172.21.15.120:0/3495058307 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda40117bd0 con 0x7fda40065c10 2023-06-23T01:18:35.070 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.058+0000 7fda3d7fa700 1 -- 172.21.15.120:0/3495058307 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3400e070 con 0x7fda40065c10 2023-06-23T01:18:35.070 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.059+0000 7fda4599d700 1 -- 172.21.15.120:0/3495058307 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda40117fd0 con 0x7fda40065c10 2023-06-23T01:18:35.070 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.059+0000 7fda4599d700 1 -- 172.21.15.120:0/3495058307 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda40108320 con 0x7fda40065c10 2023-06-23T01:18:35.070 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.061+0000 7fda3d7fa700 1 -- 172.21.15.120:0/3495058307 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7fda3401c430 con 0x7fda40065c10 2023-06-23T01:18:35.071 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.062+0000 7fda3d7fa700 1 --2- 172.21.15.120:0/3495058307 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fda3009b680 0x7fda3009db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:35.071 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.062+0000 7fda3d7fa700 1 -- 172.21.15.120:0/3495058307 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7fda340c5a80 con 0x7fda40065c10 2023-06-23T01:18:35.072 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.062+0000 7fda3ffff700 1 --2- 172.21.15.120:0/3495058307 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fda3009b680 0x7fda3009db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:35.072 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.063+0000 7fda3ffff700 1 --2- 172.21.15.120:0/3495058307 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fda3009b680 0x7fda3009db30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fda2c009b80 tx=0x7fda2c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:35.072 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.066+0000 7fda3d7fa700 1 -- 172.21.15.120:0/3495058307 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fda340916e0 con 0x7fda40065c10 2023-06-23T01:18:35.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:35 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:35.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:35 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/1941468764' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T01:18:35.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:35 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/533398391' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-06-23T01:18:35.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:35 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:35.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:35 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:35.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:35 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:35.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:35 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:35.272 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:35.269+0000 7fda4599d700 1 -- 172.21.15.120:0/3495058307 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fda401084b0 con 0x7fda40065c10 2023-06-23T01:18:35.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:35 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:35.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:35 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/1941468764' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T01:18:35.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:35 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/533398391' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-06-23T01:18:35.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:35 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:35.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:35 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:35.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:35 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:35.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:35 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:35.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:35 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:35.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:35 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/1941468764' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T01:18:35.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:35 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/533398391' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-06-23T01:18:35.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:35 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:35.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:35 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:35.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:35 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:35.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:35 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:36.062 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.059+0000 7fda3d7fa700 1 -- 172.21.15.120:0/3495058307 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) v1 ==== 138+0+0 (secure 0 0 0) 0x7fda401084b0 con 0x7fda40065c10 2023-06-23T01:18:36.068 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.064+0000 7fda4599d700 1 -- 172.21.15.120:0/3495058307 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fda3009b680 msgr2=0x7fda3009db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:36.068 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.064+0000 7fda4599d700 1 --2- 172.21.15.120:0/3495058307 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fda3009b680 0x7fda3009db30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fda2c009b80 tx=0x7fda2c00b040 comp rx=0 tx=0).stop 2023-06-23T01:18:36.068 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.064+0000 7fda4599d700 1 -- 172.21.15.120:0/3495058307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda40065c10 msgr2=0x7fda4010fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:36.069 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.064+0000 7fda4599d700 1 --2- 172.21.15.120:0/3495058307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda40065c10 0x7fda4010fa40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fda34009150 tx=0x7fda34009e30 comp rx=0 tx=0).stop 2023-06-23T01:18:36.069 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.066+0000 7fda4599d700 1 -- 172.21.15.120:0/3495058307 shutdown_connections 2023-06-23T01:18:36.070 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.066+0000 7fda4599d700 1 --2- 172.21.15.120:0/3495058307 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fda3009b680 0x7fda3009db30 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:36.070 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.066+0000 7fda4599d700 1 --2- 172.21.15.120:0/3495058307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda40102740 0x7fda40117010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:36.070 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.066+0000 7fda4599d700 1 --2- 172.21.15.120:0/3495058307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda40065c10 0x7fda4010fa40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:36.071 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.066+0000 7fda4599d700 1 --2- 172.21.15.120:0/3495058307 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda400ffc50 0x7fda4010ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:36.071 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.066+0000 7fda4599d700 1 -- 172.21.15.120:0/3495058307 >> 172.21.15.120:0/3495058307 conn(0x7fda400f7100 msgr2=0x7fda400f94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:36.071 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.066+0000 7fda4599d700 1 -- 172.21.15.120:0/3495058307 shutdown_connections 2023-06-23T01:18:36.072 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.066+0000 7fda4599d700 1 -- 172.21.15.120:0/3495058307 wait complete. 2023-06-23T01:18:36.072 INFO:teuthology.orchestra.run.smithi120.stderr:pool 'cephfs_metadata' created 2023-06-23T01:18:36.089 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-06-23T01:18:36.287 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.283+0000 7f8f6499d700 1 -- 172.21.15.120:0/3437550922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f600fe680 msgr2=0x7f8f60104f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:36.287 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.283+0000 7f8f6499d700 1 --2- 172.21.15.120:0/3437550922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f600fe680 0x7f8f60104f40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8f58004ca0 tx=0x7f8f58009ca0 comp rx=0 tx=0).stop 2023-06-23T01:18:36.287 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.283+0000 7f8f6499d700 1 -- 172.21.15.120:0/3437550922 shutdown_connections 2023-06-23T01:18:36.287 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.283+0000 7f8f6499d700 1 --2- 172.21.15.120:0/3437550922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f600fe680 0x7f8f60104f40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:36.288 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.283+0000 7f8f6499d700 1 --2- 172.21.15.120:0/3437550922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f600fcae0 0x7f8f600fcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:36.288 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.283+0000 7f8f6499d700 1 --2- 172.21.15.120:0/3437550922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f600fdcf0 0x7f8f600fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:36.288 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.283+0000 7f8f6499d700 1 -- 172.21.15.120:0/3437550922 >> 172.21.15.120:0/3437550922 conn(0x7f8f600f8200 msgr2=0x7f8f600fa670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:36.288 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.283+0000 7f8f6499d700 1 -- 172.21.15.120:0/3437550922 shutdown_connections 2023-06-23T01:18:36.288 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.283+0000 7f8f6499d700 1 -- 172.21.15.120:0/3437550922 wait complete. 2023-06-23T01:18:36.288 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.284+0000 7f8f6499d700 1 Processor -- start 2023-06-23T01:18:36.290 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.284+0000 7f8f6499d700 1 -- start start 2023-06-23T01:18:36.291 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.284+0000 7f8f6499d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f600fcae0 0x7f8f60071720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:36.291 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.284+0000 7f8f6499d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f600fdcf0 0x7f8f6006fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:36.291 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.284+0000 7f8f6499d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f600fe680 0x7f8f600702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:36.291 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.284+0000 7f8f6499d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f60070a30 con 0x7f8f600fcae0 2023-06-23T01:18:36.291 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.284+0000 7f8f6499d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f60070ba0 con 0x7f8f600fdcf0 2023-06-23T01:18:36.291 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.284+0000 7f8f6499d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f60117680 con 0x7f8f600fe680 2023-06-23T01:18:36.291 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.285+0000 7f8f5f7fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f600fcae0 0x7f8f60071720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:36.292 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.285+0000 7f8f5effd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f600fdcf0 0x7f8f6006fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:36.292 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.285+0000 7f8f5ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f600fe680 0x7f8f600702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:36.292 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.285+0000 7f8f5effd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f600fdcf0 0x7f8f6006fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:51388/0 (socket says 172.21.15.120:51388) 2023-06-23T01:18:36.292 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.285+0000 7f8f5effd700 1 -- 172.21.15.120:0/3630119847 learned_addr learned my addr 172.21.15.120:0/3630119847 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:18:36.292 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.285+0000 7f8f5effd700 1 -- 172.21.15.120:0/3630119847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f600fe680 msgr2=0x7f8f600702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:36.292 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.285+0000 7f8f5effd700 1 --2- 172.21.15.120:0/3630119847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f600fe680 0x7f8f600702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:36.292 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.285+0000 7f8f5effd700 1 -- 172.21.15.120:0/3630119847 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f600fcae0 msgr2=0x7f8f60071720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:36.293 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.285+0000 7f8f5effd700 1 --2- 172.21.15.120:0/3630119847 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f600fcae0 0x7f8f60071720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:36.293 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.285+0000 7f8f5effd700 1 -- 172.21.15.120:0/3630119847 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f54005040 con 0x7f8f600fdcf0 2023-06-23T01:18:36.293 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.285+0000 7f8f5ffff700 1 --2- 172.21.15.120:0/3630119847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f600fe680 0x7f8f600702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T01:18:36.293 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.285+0000 7f8f5effd700 1 --2- 172.21.15.120:0/3630119847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f600fdcf0 0x7f8f6006fd80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8f54009730 tx=0x7f8f54009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:36.293 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.286+0000 7f8f47fff700 1 -- 172.21.15.120:0/3630119847 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f54007be0 con 0x7f8f600fdcf0 2023-06-23T01:18:36.293 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.286+0000 7f8f47fff700 1 -- 172.21.15.120:0/3630119847 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f5400f040 con 0x7f8f600fdcf0 2023-06-23T01:18:36.293 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.286+0000 7f8f47fff700 1 -- 172.21.15.120:0/3630119847 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f54013bc0 con 0x7f8f600fdcf0 2023-06-23T01:18:36.294 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.286+0000 7f8f6499d700 1 -- 172.21.15.120:0/3630119847 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f580048f0 con 0x7f8f600fdcf0 2023-06-23T01:18:36.294 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.286+0000 7f8f6499d700 1 -- 172.21.15.120:0/3630119847 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f60117cb0 con 0x7f8f600fdcf0 2023-06-23T01:18:36.294 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.287+0000 7f8f6499d700 1 -- 172.21.15.120:0/3630119847 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f60108330 con 0x7f8f600fdcf0 2023-06-23T01:18:36.294 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.290+0000 7f8f47fff700 1 -- 172.21.15.120:0/3630119847 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7f8f54042bb0 con 0x7f8f600fdcf0 2023-06-23T01:18:36.295 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.292+0000 7f8f47fff700 1 --2- 172.21.15.120:0/3630119847 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f8f4809b700 0x7f8f4809dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:36.295 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.292+0000 7f8f47fff700 1 -- 172.21.15.120:0/3630119847 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7493+0+0 (secure 0 0 0) 0x7f8f540c4cd0 con 0x7f8f600fdcf0 2023-06-23T01:18:36.295 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.292+0000 7f8f5f7fe700 1 --2- 172.21.15.120:0/3630119847 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f8f4809b700 0x7f8f4809dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:36.296 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.292+0000 7f8f5f7fe700 1 --2- 172.21.15.120:0/3630119847 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f8f4809b700 0x7f8f4809dbb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8f50009b80 tx=0x7f8f5000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:36.296 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.293+0000 7f8f47fff700 1 -- 172.21.15.120:0/3630119847 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f8f540907c0 con 0x7f8f600fdcf0 2023-06-23T01:18:36.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:36 smithi157 ceph-mon[116945]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:36.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:36 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/3495058307' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-06-23T01:18:36.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:36 smithi157 ceph-mon[116945]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-06-23T01:18:36.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:36 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:36.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:36 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:36.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:36 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:18:36.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:36 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:18:36.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:36 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:36.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:36 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:18:36.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:36 smithi134 ceph-mon[116146]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:36.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:36 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/3495058307' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-06-23T01:18:36.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:36 smithi134 ceph-mon[116146]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-06-23T01:18:36.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:36 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:36.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:36 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:36.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:36 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:18:36.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:36 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:18:36.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:36 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:36.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:36 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:18:36.500 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:36.498+0000 7f8f6499d700 1 -- 172.21.15.120:0/3630119847 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8f60103750 con 0x7f8f600fdcf0 2023-06-23T01:18:36.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:36 smithi120 ceph-mon[79813]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:36.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:36 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/3495058307' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-06-23T01:18:36.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:36 smithi120 ceph-mon[79813]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-06-23T01:18:36.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:36 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:36.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:36 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:36.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:36 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T01:18:36.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:36 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T01:18:36.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:36 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:36.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:36 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T01:18:37.074 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.068+0000 7f8f47fff700 1 -- 172.21.15.120:0/3630119847 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v83) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8f54042e60 con 0x7f8f600fdcf0 2023-06-23T01:18:37.080 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.077+0000 7f8f6499d700 1 -- 172.21.15.120:0/3630119847 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f8f4809b700 msgr2=0x7f8f4809dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:37.080 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.077+0000 7f8f6499d700 1 --2- 172.21.15.120:0/3630119847 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f8f4809b700 0x7f8f4809dbb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8f50009b80 tx=0x7f8f5000b040 comp rx=0 tx=0).stop 2023-06-23T01:18:37.080 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.077+0000 7f8f6499d700 1 -- 172.21.15.120:0/3630119847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f600fdcf0 msgr2=0x7f8f6006fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:37.081 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.077+0000 7f8f6499d700 1 --2- 172.21.15.120:0/3630119847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f600fdcf0 0x7f8f6006fd80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8f54009730 tx=0x7f8f54009e30 comp rx=0 tx=0).stop 2023-06-23T01:18:37.081 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.077+0000 7f8f6499d700 1 -- 172.21.15.120:0/3630119847 shutdown_connections 2023-06-23T01:18:37.081 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.077+0000 7f8f6499d700 1 --2- 172.21.15.120:0/3630119847 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f8f4809b700 0x7f8f4809dbb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:37.081 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.077+0000 7f8f6499d700 1 --2- 172.21.15.120:0/3630119847 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f600fcae0 0x7f8f60071720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:37.081 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.077+0000 7f8f6499d700 1 --2- 172.21.15.120:0/3630119847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f600fe680 0x7f8f600702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:37.081 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.077+0000 7f8f6499d700 1 --2- 172.21.15.120:0/3630119847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f600fdcf0 0x7f8f6006fd80 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:37.081 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.077+0000 7f8f6499d700 1 -- 172.21.15.120:0/3630119847 >> 172.21.15.120:0/3630119847 conn(0x7f8f600f8200 msgr2=0x7f8f600fa210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:37.081 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.077+0000 7f8f6499d700 1 -- 172.21.15.120:0/3630119847 shutdown_connections 2023-06-23T01:18:37.082 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.077+0000 7f8f6499d700 1 -- 172.21.15.120:0/3630119847 wait complete. 2023-06-23T01:18:37.082 INFO:teuthology.orchestra.run.smithi120.stderr:pool 'cephfs_data' created 2023-06-23T01:18:37.108 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-06-23T01:18:37.305 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.302+0000 7fc0bf59e700 1 -- 172.21.15.120:0/4077126529 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0c0102870 msgr2=0x7fc0c0104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:37.305 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.302+0000 7fc0bf59e700 1 --2- 172.21.15.120:0/4077126529 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0c0102870 0x7fc0c0104c50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8004ca0 tx=0x7fc0b8009ca0 comp rx=0 tx=0).stop 2023-06-23T01:18:37.305 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.302+0000 7fc0bf59e700 1 -- 172.21.15.120:0/4077126529 shutdown_connections 2023-06-23T01:18:37.306 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.302+0000 7fc0bf59e700 1 --2- 172.21.15.120:0/4077126529 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0c0102870 0x7fc0c0104c50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:37.306 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.302+0000 7fc0bf59e700 1 --2- 172.21.15.120:0/4077126529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0c00fd630 0x7fc0c00ffa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:37.306 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.302+0000 7fc0bf59e700 1 --2- 172.21.15.120:0/4077126529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0c00fff50 0x7fc0c0102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:37.306 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.302+0000 7fc0bf59e700 1 -- 172.21.15.120:0/4077126529 >> 172.21.15.120:0/4077126529 conn(0x7fc0c00f7150 msgr2=0x7fc0c00f9580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:37.306 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.302+0000 7fc0bf59e700 1 -- 172.21.15.120:0/4077126529 shutdown_connections 2023-06-23T01:18:37.306 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.302+0000 7fc0bf59e700 1 -- 172.21.15.120:0/4077126529 wait complete. 2023-06-23T01:18:37.308 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.303+0000 7fc0bf59e700 1 Processor -- start 2023-06-23T01:18:37.308 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.303+0000 7fc0bf59e700 1 -- start start 2023-06-23T01:18:37.308 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.303+0000 7fc0bf59e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0c00fd630 0x7fc0c00fcee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:37.308 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.303+0000 7fc0bf59e700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0c00fff50 0x7fc0c00fb540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:37.309 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.303+0000 7fc0bf59e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0c0102870 0x7fc0c00fba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:37.309 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.303+0000 7fc0bf59e700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c00fc1f0 con 0x7fc0c00fff50 2023-06-23T01:18:37.309 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.303+0000 7fc0bf59e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c00fc360 con 0x7fc0c0102870 2023-06-23T01:18:37.309 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.303+0000 7fc0bf59e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c0117720 con 0x7fc0c00fd630 2023-06-23T01:18:37.309 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.304+0000 7fc0be59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0c00fd630 0x7fc0c00fcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:37.309 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.304+0000 7fc0be59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0c00fd630 0x7fc0c00fcee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.120:49466/0 (socket says 172.21.15.120:49466) 2023-06-23T01:18:37.309 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.304+0000 7fc0be59c700 1 -- 172.21.15.120:0/1138423833 learned_addr learned my addr 172.21.15.120:0/1138423833 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:18:37.310 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.304+0000 7fc0bdd9b700 1 --2- 172.21.15.120:0/1138423833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0c00fff50 0x7fc0c00fb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:37.310 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.304+0000 7fc0bed9d700 1 --2- 172.21.15.120:0/1138423833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0c0102870 0x7fc0c00fba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:37.310 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.304+0000 7fc0be59c700 1 -- 172.21.15.120:0/1138423833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0c0102870 msgr2=0x7fc0c00fba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:37.310 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.304+0000 7fc0be59c700 1 --2- 172.21.15.120:0/1138423833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0c0102870 0x7fc0c00fba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:37.310 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.304+0000 7fc0be59c700 1 -- 172.21.15.120:0/1138423833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0c00fff50 msgr2=0x7fc0c00fb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:37.311 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.304+0000 7fc0be59c700 1 --2- 172.21.15.120:0/1138423833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0c00fff50 0x7fc0c00fb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:37.311 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.304+0000 7fc0be59c700 1 -- 172.21.15.120:0/1138423833 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b80048f0 con 0x7fc0c00fd630 2023-06-23T01:18:37.311 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.304+0000 7fc0bdd9b700 1 --2- 172.21.15.120:0/1138423833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0c00fff50 0x7fc0c00fb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T01:18:37.311 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.304+0000 7fc0be59c700 1 --2- 172.21.15.120:0/1138423833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0c00fd630 0x7fc0c00fcee0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc0b00041d0 tx=0x7fc0b000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:37.311 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.305+0000 7fc0aeffd700 1 -- 172.21.15.120:0/1138423833 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b0007cc0 con 0x7fc0c00fd630 2023-06-23T01:18:37.311 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.305+0000 7fc0bf59e700 1 -- 172.21.15.120:0/1138423833 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0c0117970 con 0x7fc0c00fd630 2023-06-23T01:18:37.311 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.305+0000 7fc0aeffd700 1 -- 172.21.15.120:0/1138423833 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0b000f040 con 0x7fc0c00fd630 2023-06-23T01:18:37.311 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.305+0000 7fc0aeffd700 1 -- 172.21.15.120:0/1138423833 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b0009540 con 0x7fc0c00fd630 2023-06-23T01:18:37.312 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.305+0000 7fc0bf59e700 1 -- 172.21.15.120:0/1138423833 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0c0117f30 con 0x7fc0c00fd630 2023-06-23T01:18:37.312 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.306+0000 7fc0bf59e700 1 -- 172.21.15.120:0/1138423833 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0c00f8c80 con 0x7fc0c00fd630 2023-06-23T01:18:37.313 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.308+0000 7fc0aeffd700 1 -- 172.21.15.120:0/1138423833 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7fc0b000c680 con 0x7fc0c00fd630 2023-06-23T01:18:37.313 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.309+0000 7fc0aeffd700 1 --2- 172.21.15.120:0/1138423833 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fc0a809b680 0x7fc0a809db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:37.313 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.309+0000 7fc0aeffd700 1 -- 172.21.15.120:0/1138423833 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7875+0+0 (secure 0 0 0) 0x7fc0b00c4990 con 0x7fc0c00fd630 2023-06-23T01:18:37.313 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.309+0000 7fc0bdd9b700 1 --2- 172.21.15.120:0/1138423833 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fc0a809b680 0x7fc0a809db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:37.313 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.310+0000 7fc0bdd9b700 1 --2- 172.21.15.120:0/1138423833 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fc0a809b680 0x7fc0a809db30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc0b4004600 tx=0x7fc0b400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:37.314 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.311+0000 7fc0aeffd700 1 -- 172.21.15.120:0/1138423833 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fc0b004da00 con 0x7fc0c00fd630 2023-06-23T01:18:37.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:37 smithi157 ceph-mon[116945]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-06-23T01:18:37.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:37 smithi157 ceph-mon[116945]: osdmap e82: 12 total, 12 up, 12 in 2023-06-23T01:18:37.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:37 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/3630119847' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-06-23T01:18:37.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:37 smithi157 ceph-mon[116945]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-06-23T01:18:37.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:37 smithi134 ceph-mon[116146]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-06-23T01:18:37.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:37 smithi134 ceph-mon[116146]: osdmap e82: 12 total, 12 up, 12 in 2023-06-23T01:18:37.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:37 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/3630119847' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-06-23T01:18:37.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:37 smithi134 ceph-mon[116146]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-06-23T01:18:37.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:37 smithi120 ceph-mon[79813]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-06-23T01:18:37.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:37 smithi120 ceph-mon[79813]: osdmap e82: 12 total, 12 up, 12 in 2023-06-23T01:18:37.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:37 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/3630119847' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-06-23T01:18:37.512 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:37 smithi120 ceph-mon[79813]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-06-23T01:18:37.584 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.581+0000 7fc0bf59e700 1 -- 172.21.15.120:0/1138423833 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc0c0108320 con 0x7fc0c00fd630 2023-06-23T01:18:37.593 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.590+0000 7fc0aeffd700 1 -- 172.21.15.120:0/1138423833 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7fc0b0019070 con 0x7fc0c00fd630 2023-06-23T01:18:37.598 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.594+0000 7fc0bf59e700 1 -- 172.21.15.120:0/1138423833 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fc0a809b680 msgr2=0x7fc0a809db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:37.598 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.594+0000 7fc0bf59e700 1 --2- 172.21.15.120:0/1138423833 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fc0a809b680 0x7fc0a809db30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc0b4004600 tx=0x7fc0b400b040 comp rx=0 tx=0).stop 2023-06-23T01:18:37.598 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.595+0000 7fc0bf59e700 1 -- 172.21.15.120:0/1138423833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0c00fd630 msgr2=0x7fc0c00fcee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:37.599 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.595+0000 7fc0bf59e700 1 --2- 172.21.15.120:0/1138423833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0c00fd630 0x7fc0c00fcee0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc0b00041d0 tx=0x7fc0b000b040 comp rx=0 tx=0).stop 2023-06-23T01:18:37.605 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.602+0000 7fc0bf59e700 1 -- 172.21.15.120:0/1138423833 shutdown_connections 2023-06-23T01:18:37.605 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.602+0000 7fc0bf59e700 1 --2- 172.21.15.120:0/1138423833 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fc0a809b680 0x7fc0a809db30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:37.605 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.602+0000 7fc0bf59e700 1 --2- 172.21.15.120:0/1138423833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0c0102870 0x7fc0c00fba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:37.605 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.602+0000 7fc0bf59e700 1 --2- 172.21.15.120:0/1138423833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0c00fd630 0x7fc0c00fcee0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:37.605 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.602+0000 7fc0bf59e700 1 --2- 172.21.15.120:0/1138423833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0c00fff50 0x7fc0c00fb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:37.606 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.602+0000 7fc0bf59e700 1 -- 172.21.15.120:0/1138423833 >> 172.21.15.120:0/1138423833 conn(0x7fc0c00f7150 msgr2=0x7fc0c00fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:37.611 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.608+0000 7fc0bf59e700 1 -- 172.21.15.120:0/1138423833 shutdown_connections 2023-06-23T01:18:37.613 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.610+0000 7fc0bf59e700 1 -- 172.21.15.120:0/1138423833 wait complete. 2023-06-23T01:18:37.620 INFO:teuthology.orchestra.run.smithi120.stderr:new fs with metadata pool 2 and data pool 3 2023-06-23T01:18:37.642 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T01:18:37.828 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.825+0000 7fd9f9b9d700 1 -- 172.21.15.120:0/986756620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9f4102750 msgr2=0x7fd9f4104c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:37.829 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.825+0000 7fd9f9b9d700 1 --2- 172.21.15.120:0/986756620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9f4102750 0x7fd9f4104c00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec004ca0 tx=0x7fd9ec009ca0 comp rx=0 tx=0).stop 2023-06-23T01:18:37.829 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.825+0000 7fd9f9b9d700 1 -- 172.21.15.120:0/986756620 shutdown_connections 2023-06-23T01:18:37.829 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.825+0000 7fd9f9b9d700 1 --2- 172.21.15.120:0/986756620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9f4066bb0 0x7fd9f40ff6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:37.829 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.825+0000 7fd9f9b9d700 1 --2- 172.21.15.120:0/986756620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9f4102750 0x7fd9f4104c00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:37.830 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.825+0000 7fd9f9b9d700 1 --2- 172.21.15.120:0/986756620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9f40ffc00 0x7fd9f4102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:37.830 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.825+0000 7fd9f9b9d700 1 -- 172.21.15.120:0/986756620 >> 172.21.15.120:0/986756620 conn(0x7fd9f40f7130 msgr2=0x7fd9f40f9560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:37.830 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.825+0000 7fd9f9b9d700 1 -- 172.21.15.120:0/986756620 shutdown_connections 2023-06-23T01:18:37.830 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.826+0000 7fd9f9b9d700 1 -- 172.21.15.120:0/986756620 wait complete. 2023-06-23T01:18:37.832 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.826+0000 7fd9f9b9d700 1 Processor -- start 2023-06-23T01:18:37.832 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.826+0000 7fd9f9b9d700 1 -- start start 2023-06-23T01:18:37.833 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.827+0000 7fd9f9b9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9f4066bb0 0x7fd9f410f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:37.833 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.827+0000 7fd9f9b9d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9f40ffc00 0x7fd9f410fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:37.833 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.827+0000 7fd9f9b9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9f4102750 0x7fd9f4116dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:37.833 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.827+0000 7fd9f9b9d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f41173d0 con 0x7fd9f40ffc00 2023-06-23T01:18:37.833 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.827+0000 7fd9f9b9d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f4117540 con 0x7fd9f4066bb0 2023-06-23T01:18:37.833 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.827+0000 7fd9f9b9d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f4117700 con 0x7fd9f4102750 2023-06-23T01:18:37.834 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.827+0000 7fd9f8b9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9f4066bb0 0x7fd9f410f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:37.834 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.827+0000 7fd9f3fff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9f40ffc00 0x7fd9f410fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:37.834 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.827+0000 7fd9f8b9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9f4066bb0 0x7fd9f410f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:51430/0 (socket says 172.21.15.120:51430) 2023-06-23T01:18:37.834 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.827+0000 7fd9f8b9b700 1 -- 172.21.15.120:0/584053984 learned_addr learned my addr 172.21.15.120:0/584053984 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:18:37.834 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.827+0000 7fd9f939c700 1 --2- 172.21.15.120:0/584053984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9f4102750 0x7fd9f4116dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:37.835 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.827+0000 7fd9f8b9b700 1 -- 172.21.15.120:0/584053984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9f4102750 msgr2=0x7fd9f4116dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:37.835 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.827+0000 7fd9f8b9b700 1 --2- 172.21.15.120:0/584053984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9f4102750 0x7fd9f4116dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:37.835 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.827+0000 7fd9f8b9b700 1 -- 172.21.15.120:0/584053984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9f40ffc00 msgr2=0x7fd9f410fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:37.835 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.827+0000 7fd9f8b9b700 1 --2- 172.21.15.120:0/584053984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9f40ffc00 0x7fd9f410fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:37.835 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.827+0000 7fd9f8b9b700 1 -- 172.21.15.120:0/584053984 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e0005040 con 0x7fd9f4066bb0 2023-06-23T01:18:37.835 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.827+0000 7fd9f8b9b700 1 --2- 172.21.15.120:0/584053984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9f4066bb0 0x7fd9f410f7f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd9e800ebc0 tx=0x7fd9e800b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:37.835 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.828+0000 7fd9f17fa700 1 -- 172.21.15.120:0/584053984 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e8007cc0 con 0x7fd9f4066bb0 2023-06-23T01:18:37.836 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.828+0000 7fd9f9b9d700 1 -- 172.21.15.120:0/584053984 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9ec0048f0 con 0x7fd9f4066bb0 2023-06-23T01:18:37.836 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.828+0000 7fd9f17fa700 1 -- 172.21.15.120:0/584053984 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9e800f040 con 0x7fd9f4066bb0 2023-06-23T01:18:37.836 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.828+0000 7fd9f17fa700 1 -- 172.21.15.120:0/584053984 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e800c8b0 con 0x7fd9f4066bb0 2023-06-23T01:18:37.836 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.828+0000 7fd9f9b9d700 1 -- 172.21.15.120:0/584053984 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9f4117db0 con 0x7fd9f4066bb0 2023-06-23T01:18:37.836 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.829+0000 7fd9f9b9d700 1 -- 172.21.15.120:0/584053984 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9f4108300 con 0x7fd9f4066bb0 2023-06-23T01:18:37.837 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.830+0000 7fd9f17fa700 1 -- 172.21.15.120:0/584053984 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7fd9e800cc60 con 0x7fd9f4066bb0 2023-06-23T01:18:37.837 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.832+0000 7fd9f17fa700 1 --2- 172.21.15.120:0/584053984 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fd9e409b680 0x7fd9e409db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:37.837 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.832+0000 7fd9f17fa700 1 -- 172.21.15.120:0/584053984 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7975+0+0 (secure 0 0 0) 0x7fd9e80c4710 con 0x7fd9f4066bb0 2023-06-23T01:18:37.837 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.832+0000 7fd9f3fff700 1 --2- 172.21.15.120:0/584053984 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fd9e409b680 0x7fd9e409db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:37.837 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.833+0000 7fd9f3fff700 1 --2- 172.21.15.120:0/584053984 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fd9e409b680 0x7fd9e409db30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0009fd0 tx=0x7fd9e000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:37.838 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:37.835+0000 7fd9f17fa700 1 -- 172.21.15.120:0/584053984 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fd9e8090020 con 0x7fd9f4066bb0 2023-06-23T01:18:38.048 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.045+0000 7fd9f9b9d700 1 -- 172.21.15.120:0/584053984 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9f4108490 con 0x7fd9f4066bb0 2023-06-23T01:18:38.049 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.046+0000 7fd9f17fa700 1 -- 172.21.15.120:0/584053984 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+21479 (secure 0 0 0) 0x7fd9f4108490 con 0x7fd9f4066bb0 2023-06-23T01:18:38.050 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:18:38.050 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":84,"fsid":"9efbfa8e-1162-11ee-9b2b-001a4aab830c","created":"2023-06-23T01:10:46.840474+0000","modified":"2023-06-23T01:18:37.582883+0000","last_up_change":"2023-06-23T01:17:46.670887+0000","last_in_change":"2023-06-23T01:17:27.552330+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T01:14:03.589464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-06-23T01:18:35.785142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-06-23T01:18:37.012022+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3729642b-e74f-4253-bde8-535d8bfe8c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3479894866},{"type":"v1","addr":"172.21.15.120:6803","nonce":3479894866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3479894866},{"type":"v1","addr":"172.21.15.120:6805","nonce":3479894866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3479894866},{"type":"v1","addr":"172.21.15.120:6809","nonce":3479894866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3479894866},{"type":"v1","addr":"172.21.15.120:6807","nonce":3479894866}]},"public_addr":"172.21.15.120:6803/3479894866","cluster_addr":"172.21.15.120:6805/3479894866","heartbeat_back_addr":"172.21.15.120:6809/3479894866","heartbeat_front_addr":"172.21.15.120:6807/3479894866","state":["exists","up"]},{"osd":1,"uuid":"58240258-afd5-4991-9d5b-a80d365e4fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3046662876},{"type":"v1","addr":"172.21.15.134:6801","nonce":3046662876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3046662876},{"type":"v1","addr":"172.21.15.134:6803","nonce":3046662876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3046662876},{"type":"v1","addr":"172.21.15.134:6807","nonce":3046662876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3046662876},{"type":"v1","addr":"172.21.15.134:6805","nonce":3046662876}]},"public_addr":"172.21.15.134:6801/3046662876","cluster_addr":"172.21.15.134:6803/3046662876","heartbeat_back_addr":"172.21.15.134:6807/3046662876","heartbeat_front_addr":"172.21.15.134:6805/3046662876","state":["exists","up"]},{"osd":2,"uuid":"05988a7a-7cf2-4fbb-9d4f-22604cd2f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3996265223},{"type":"v1","addr":"172.21.15.157:6801","nonce":3996265223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3996265223},{"type":"v1","addr":"172.21.15.157:6803","nonce":3996265223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3996265223},{"type":"v1","addr":"172.21.15.157:6807","nonce":3996265223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3996265223},{"type":"v1","addr":"172.21.15.157:6805","nonce":3996265223}]},"public_addr":"172.21.15.157:6801/3996265223","cluster_addr":"172.21.15.157:6803/3996265223","heartbeat_back_addr":"172.21.15.157:6807/3996265223","heartbeat_front_addr":"172.21.15.157:6805/3996265223","state":["exists","up"]},{"osd":3,"uuid":"7d9e9739-f043-43d6-8e46-7711dcf43e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":1716638437},{"type":"v1","addr":"172.21.15.120:6811","nonce":1716638437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":1716638437},{"type":"v1","addr":"172.21.15.120:6813","nonce":1716638437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1716638437},{"type":"v1","addr":"172.21.15.120:6817","nonce":1716638437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":1716638437},{"type":"v1","addr":"172.21.15.120:6815","nonce":1716638437}]},"public_addr":"172.21.15.120:6811/1716638437","cluster_addr":"172.21.15.120:6813/1716638437","heartbeat_back_addr":"172.21.15.120:6817/1716638437","heartbeat_front_addr":"172.21.15.120:6815/1716638437","state":["exists","up"]},{"osd":4,"uuid":"3a70f86d-f903-4618-be66-9aa1eac44ccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4172238262},{"type":"v1","addr":"172.21.15.134:6809","nonce":4172238262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4172238262},{"type":"v1","addr":"172.21.15.134:6811","nonce":4172238262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4172238262},{"type":"v1","addr":"172.21.15.134:6815","nonce":4172238262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4172238262},{"type":"v1","addr":"172.21.15.134:6813","nonce":4172238262}]},"public_addr":"172.21.15.134:6809/4172238262","cluster_addr":"172.21.15.134:6811/4172238262","heartbeat_back_addr":"172.21.15.134:6815/4172238262","heartbeat_front_addr":"172.21.15.134:6813/4172238262","state":["exists","up"]},{"osd":5,"uuid":"b2316488-75c3-4847-b4c4-998730f27c88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3374822045},{"type":"v1","addr":"172.21.15.157:6809","nonce":3374822045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3374822045},{"type":"v1","addr":"172.21.15.157:6811","nonce":3374822045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3374822045},{"type":"v1","addr":"172.21.15.157:6815","nonce":3374822045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3374822045},{"type":"v1","addr":"172.21.15.157:6813","nonce":3374822045}]},"public_addr":"172.21.15.157:6809/3374822045","cluster_addr":"172.21.15.157:6811/3374822045","heartbeat_back_addr":"172.21.15.157:6815/3374822045","heartbeat_front_addr":"172.21.15.157:6813/3374822045","state":["exists","up"]},{"osd":6,"uuid":"338c2c4a-d982-4031-aaaf-be76c9dd74a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2097345225},{"type":"v1","addr":"172.21.15.120:6819","nonce":2097345225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2097345225},{"type":"v1","addr":"172.21.15.120:6821","nonce":2097345225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2097345225},{"type":"v1","addr":"172.21.15.120:6825","nonce":2097345225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2097345225},{"type":"v1","addr":"172.21.15.120:6823","nonce":2097345225}]},"public_addr":"172.21.15.120:6819/2097345225","cluster_addr":"172.21.15.120:6821/2097345225","heartbeat_back_addr":"172.21.15.120:6825/2097345225","heartbeat_front_addr":"172.21.15.120:6823/2097345225","state":["exists","up"]},{"osd":7,"uuid":"bff92799-ae9c-4409-be65-7ffc27a1c3f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1345745298},{"type":"v1","addr":"172.21.15.134:6817","nonce":1345745298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1345745298},{"type":"v1","addr":"172.21.15.134:6819","nonce":1345745298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1345745298},{"type":"v1","addr":"172.21.15.134:6823","nonce":1345745298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1345745298},{"type":"v1","addr":"172.21.15.134:6821","nonce":1345745298}]},"public_addr":"172.21.15.134:6817/1345745298","cluster_addr":"172.21.15.134:6819/1345745298","heartbeat_back_addr":"172.21.15.134:6823/1345745298","heartbeat_front_addr":"172.21.15.134:6821/1345745298","state":["exists","up"]},{"osd":8,"uuid":"e4aac47e-d7f9-4feb-ada7-d8103fdca246","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1124549599},{"type":"v1","addr":"172.21.15.157:6817","nonce":1124549599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1124549599},{"type":"v1","addr":"172.21.15.157:6819","nonce":1124549599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1124549599},{"type":"v1","addr":"172.21.15.157:6823","nonce":1124549599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1124549599},{"type":"v1","addr":"172.21.15.157:6821","nonce":1124549599}]},"public_addr":"172.21.15.157:6817/1124549599","cluster_addr":"172.21.15.157:6819/1124549599","heartbeat_back_addr":"172.21.15.157:6823/1124549599","heartbeat_front_addr":"172.21.15.157:6821/1124549599","state":["exists","up"]},{"osd":9,"uuid":"48b988ba-e866-47d1-aca9-09f6ad815bd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1402414981},{"type":"v1","addr":"172.21.15.120:6827","nonce":1402414981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1402414981},{"type":"v1","addr":"172.21.15.120:6829","nonce":1402414981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1402414981},{"type":"v1","addr":"172.21.15.120:6833","nonce":1402414981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1402414981},{"type":"v1","addr":"172.21.15.120:6831","nonce":1402414981}]},"public_addr":"172.21.15.120:6827/1402414981","cluster_addr":"172.21.15.120:6829/1402414981","heartbeat_back_addr":"172.21.15.120:6833/1402414981","heartbeat_front_addr":"172.21.15.120:6831/1402414981","state":["exists","up"]},{"osd":10,"uuid":"777f3add-3779-4221-a86f-65cca2e5e8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":4041441144},{"type":"v1","addr":"172.21.15.134:6825","nonce":4041441144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":4041441144},{"type":"v1","addr":"172.21.15.134:6827","nonce":4041441144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":4041441144},{"type":"v1","addr":"172.21.15.134:6831","nonce":4041441144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":4041441144},{"type":"v1","addr":"172.21.15.134:6829","nonce":4041441144}]},"public_addr":"172.21.15.134:6825/4041441144","cluster_addr":"172.21.15.134:6827/4041441144","heartbeat_back_addr":"172.21.15.134:6831/4041441144","heartbeat_front_addr":"172.21.15.134:6829/4041441144","state":["exists","up"]},{"osd":11,"uuid":"7166e11b-03c0-45bf-abbf-a74c4a13a39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1537477260},{"type":"v1","addr":"172.21.15.157:6825","nonce":1537477260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1537477260},{"type":"v1","addr":"172.21.15.157:6827","nonce":1537477260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1537477260},{"type":"v1","addr":"172.21.15.157:6831","nonce":1537477260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1537477260},{"type":"v1","addr":"172.21.15.157:6829","nonce":1537477260}]},"public_addr":"172.21.15.157:6825/1537477260","cluster_addr":"172.21.15.157:6827/1537477260","heartbeat_back_addr":"172.21.15.157:6831/1537477260","heartbeat_front_addr":"172.21.15.157:6829/1537477260","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:13:13.580631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:13:37.472924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:14:01.591555+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:14:28.472130+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:14:52.356433+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:15:15.946523+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:15:42.036105+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:16:06.154934+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:16:30.043788+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:16:55.975403+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:17:20.612778+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:17:44.896301+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.120:0/2865552846":"2023-06-24T01:11:38.967204+0000","172.21.15.120:0/2163781512":"2023-06-24T01:11:38.967204+0000","172.21.15.120:0/1943345772":"2023-06-24T01:11:38.967204+0000","172.21.15.120:6801/1836713308":"2023-06-24T01:11:38.967204+0000","172.21.15.120:0/2358086231":"2023-06-24T01:11:14.574788+0000","172.21.15.120:6801/506843621":"2023-06-24T01:11:14.574788+0000","172.21.15.120:6800/506843621":"2023-06-24T01:11:14.574788+0000","172.21.15.120:0/4077038498":"2023-06-24T01:11:14.574788+0000","172.21.15.120:6800/1836713308":"2023-06-24T01:11:38.967204+0000","172.21.15.120:0/2484415475":"2023-06-24T01:11:14.574788+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T01:18:38.052 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.049+0000 7fd9f9b9d700 1 -- 172.21.15.120:0/584053984 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fd9e409b680 msgr2=0x7fd9e409db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:38.053 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.049+0000 7fd9f9b9d700 1 --2- 172.21.15.120:0/584053984 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fd9e409b680 0x7fd9e409db30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0009fd0 tx=0x7fd9e000b040 comp rx=0 tx=0).stop 2023-06-23T01:18:38.053 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.049+0000 7fd9f9b9d700 1 -- 172.21.15.120:0/584053984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9f4066bb0 msgr2=0x7fd9f410f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:38.053 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.049+0000 7fd9f9b9d700 1 --2- 172.21.15.120:0/584053984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9f4066bb0 0x7fd9f410f7f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd9e800ebc0 tx=0x7fd9e800b040 comp rx=0 tx=0).stop 2023-06-23T01:18:38.053 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.049+0000 7fd9f9b9d700 1 -- 172.21.15.120:0/584053984 shutdown_connections 2023-06-23T01:18:38.054 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.049+0000 7fd9f9b9d700 1 --2- 172.21.15.120:0/584053984 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7fd9e409b680 0x7fd9e409db30 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:38.054 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.049+0000 7fd9f9b9d700 1 --2- 172.21.15.120:0/584053984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9f4102750 0x7fd9f4116dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:38.054 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.049+0000 7fd9f9b9d700 1 --2- 172.21.15.120:0/584053984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9f4066bb0 0x7fd9f410f7f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:38.055 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.049+0000 7fd9f9b9d700 1 --2- 172.21.15.120:0/584053984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9f40ffc00 0x7fd9f410fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:38.055 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.049+0000 7fd9f9b9d700 1 -- 172.21.15.120:0/584053984 >> 172.21.15.120:0/584053984 conn(0x7fd9f40f7130 msgr2=0x7fd9f40f9510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:38.055 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.049+0000 7fd9f9b9d700 1 -- 172.21.15.120:0/584053984 shutdown_connections 2023-06-23T01:18:38.055 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.049+0000 7fd9f9b9d700 1 -- 172.21.15.120:0/584053984 wait complete. 2023-06-23T01:18:38.073 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T01:18:38.269 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.266+0000 7f19f5b9d700 1 -- 172.21.15.120:0/2657926274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19f00ffc70 msgr2=0x7f19f0102050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:38.269 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.266+0000 7f19f5b9d700 1 --2- 172.21.15.120:0/2657926274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19f00ffc70 0x7f19f0102050 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f19dc0049c0 tx=0x7f19dc009e30 comp rx=0 tx=0).stop 2023-06-23T01:18:38.269 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.266+0000 7f19f5b9d700 1 -- 172.21.15.120:0/2657926274 shutdown_connections 2023-06-23T01:18:38.270 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.266+0000 7f19f5b9d700 1 --2- 172.21.15.120:0/2657926274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19f0065c10 0x7f19f00ff730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:38.270 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.266+0000 7f19f5b9d700 1 --2- 172.21.15.120:0/2657926274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f0102760 0x7f19f0104c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:38.270 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.266+0000 7f19f5b9d700 1 --2- 172.21.15.120:0/2657926274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19f00ffc70 0x7f19f0102050 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:38.270 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.266+0000 7f19f5b9d700 1 -- 172.21.15.120:0/2657926274 >> 172.21.15.120:0/2657926274 conn(0x7f19f00f7140 msgr2=0x7f19f00f9570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:38.270 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.266+0000 7f19f5b9d700 1 -- 172.21.15.120:0/2657926274 shutdown_connections 2023-06-23T01:18:38.271 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.266+0000 7f19f5b9d700 1 -- 172.21.15.120:0/2657926274 wait complete. 2023-06-23T01:18:38.272 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.267+0000 7f19f5b9d700 1 Processor -- start 2023-06-23T01:18:38.272 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.267+0000 7f19f5b9d700 1 -- start start 2023-06-23T01:18:38.273 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.267+0000 7f19f5b9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f0065c10 0x7f19f00fcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:38.273 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.267+0000 7f19f5b9d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19f00ffc70 0x7f19f00fb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:38.273 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.267+0000 7f19f5b9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19f0102760 0x7f19f00fbb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:38.273 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.267+0000 7f19f5b9d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f00fc2e0 con 0x7f19f00ffc70 2023-06-23T01:18:38.274 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.267+0000 7f19f5b9d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f00fc450 con 0x7f19f0102760 2023-06-23T01:18:38.274 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.267+0000 7f19f5b9d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f01178e0 con 0x7f19f0065c10 2023-06-23T01:18:38.274 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.267+0000 7f19f4b9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f0065c10 0x7f19f00fcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:38.274 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.267+0000 7f19f4b9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f0065c10 0x7f19f00fcfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.120:49500/0 (socket says 172.21.15.120:49500) 2023-06-23T01:18:38.274 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.267+0000 7f19f539c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19f0102760 0x7f19f00fbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:38.275 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.267+0000 7f19f4b9b700 1 -- 172.21.15.120:0/2416356178 learned_addr learned my addr 172.21.15.120:0/2416356178 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:18:38.275 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.267+0000 7f19effff700 1 --2- 172.21.15.120:0/2416356178 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19f00ffc70 0x7f19f00fb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:38.275 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.268+0000 7f19f4b9b700 1 -- 172.21.15.120:0/2416356178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19f0102760 msgr2=0x7f19f00fbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:38.275 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.268+0000 7f19f4b9b700 1 --2- 172.21.15.120:0/2416356178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19f0102760 0x7f19f00fbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:38.275 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.268+0000 7f19f4b9b700 1 -- 172.21.15.120:0/2416356178 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19f00ffc70 msgr2=0x7f19f00fb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:38.275 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.268+0000 7f19f4b9b700 1 --2- 172.21.15.120:0/2416356178 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19f00ffc70 0x7f19f00fb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:38.275 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.268+0000 7f19f4b9b700 1 -- 172.21.15.120:0/2416356178 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19dc005040 con 0x7f19f0065c10 2023-06-23T01:18:38.276 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.268+0000 7f19f4b9b700 1 --2- 172.21.15.120:0/2416356178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f0065c10 0x7f19f00fcfd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f19e400ebf0 tx=0x7f19e400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:38.276 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.268+0000 7f19ed7fa700 1 -- 172.21.15.120:0/2416356178 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e4007cc0 con 0x7f19f0065c10 2023-06-23T01:18:38.276 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.268+0000 7f19f5b9d700 1 -- 172.21.15.120:0/2416356178 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f0117b30 con 0x7f19f0065c10 2023-06-23T01:18:38.276 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.268+0000 7f19ed7fa700 1 -- 172.21.15.120:0/2416356178 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19e400f040 con 0x7f19f0065c10 2023-06-23T01:18:38.276 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.268+0000 7f19ed7fa700 1 -- 172.21.15.120:0/2416356178 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e400c8f0 con 0x7f19f0065c10 2023-06-23T01:18:38.277 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.268+0000 7f19f5b9d700 1 -- 172.21.15.120:0/2416356178 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f0118020 con 0x7f19f0065c10 2023-06-23T01:18:38.277 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.269+0000 7f19f5b9d700 1 -- 172.21.15.120:0/2416356178 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19f00f8c70 con 0x7f19f0065c10 2023-06-23T01:18:38.277 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.271+0000 7f19ed7fa700 1 -- 172.21.15.120:0/2416356178 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7f19e400cd20 con 0x7f19f0065c10 2023-06-23T01:18:38.278 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.272+0000 7f19ed7fa700 1 --2- 172.21.15.120:0/2416356178 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f19e009b680 0x7f19e009db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:38.278 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.272+0000 7f19ed7fa700 1 -- 172.21.15.120:0/2416356178 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7975+0+0 (secure 0 0 0) 0x7f19e40c4760 con 0x7f19f0065c10 2023-06-23T01:18:38.278 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.272+0000 7f19effff700 1 --2- 172.21.15.120:0/2416356178 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f19e009b680 0x7f19e009db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:38.278 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.273+0000 7f19effff700 1 --2- 172.21.15.120:0/2416356178 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f19e009b680 0x7f19e009db30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f19dc004890 tx=0x7f19dc009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:38.279 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.276+0000 7f19ed7fa700 1 -- 172.21.15.120:0/2416356178 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f19e404da00 con 0x7f19f0065c10 2023-06-23T01:18:38.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:38 smithi157 ceph-mon[116945]: pgmap v281: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:38.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:38 smithi157 ceph-mon[116945]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-06-23T01:18:38.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:38 smithi157 ceph-mon[116945]: osdmap e83: 12 total, 12 up, 12 in 2023-06-23T01:18:38.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:38 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/1138423833' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-06-23T01:18:38.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:38 smithi157 ceph-mon[116945]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-06-23T01:18:38.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:38 smithi157 ceph-mon[116945]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-06-23T01:18:38.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:38 smithi157 ceph-mon[116945]: osdmap e84: 12 total, 12 up, 12 in 2023-06-23T01:18:38.379 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:38 smithi157 ceph-mon[116945]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-06-23T01:18:38.379 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:38 smithi157 ceph-mon[116945]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-06-23T01:18:38.379 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:38 smithi157 ceph-mon[116945]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-06-23T01:18:38.380 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:38 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/584053984' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T01:18:38.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:38 smithi134 ceph-mon[116146]: pgmap v281: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:38.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:38 smithi134 ceph-mon[116146]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-06-23T01:18:38.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:38 smithi134 ceph-mon[116146]: osdmap e83: 12 total, 12 up, 12 in 2023-06-23T01:18:38.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:38 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/1138423833' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-06-23T01:18:38.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:38 smithi134 ceph-mon[116146]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-06-23T01:18:38.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:38 smithi134 ceph-mon[116146]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-06-23T01:18:38.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:38 smithi134 ceph-mon[116146]: osdmap e84: 12 total, 12 up, 12 in 2023-06-23T01:18:38.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:38 smithi134 ceph-mon[116146]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-06-23T01:18:38.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:38 smithi134 ceph-mon[116146]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-06-23T01:18:38.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:38 smithi134 ceph-mon[116146]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-06-23T01:18:38.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:38 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/584053984' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T01:18:38.496 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.493+0000 7f19f5b9d700 1 -- 172.21.15.120:0/2416356178 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19f0108330 con 0x7f19f0065c10 2023-06-23T01:18:38.497 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.495+0000 7f19ed7fa700 1 -- 172.21.15.120:0/2416356178 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+21479 (secure 0 0 0) 0x7f19e4019090 con 0x7f19f0065c10 2023-06-23T01:18:38.498 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-23T01:18:38.498 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":84,"fsid":"9efbfa8e-1162-11ee-9b2b-001a4aab830c","created":"2023-06-23T01:10:46.840474+0000","modified":"2023-06-23T01:18:37.582883+0000","last_up_change":"2023-06-23T01:17:46.670887+0000","last_in_change":"2023-06-23T01:17:27.552330+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T01:14:03.589464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-06-23T01:18:35.785142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-06-23T01:18:37.012022+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3729642b-e74f-4253-bde8-535d8bfe8c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3479894866},{"type":"v1","addr":"172.21.15.120:6803","nonce":3479894866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3479894866},{"type":"v1","addr":"172.21.15.120:6805","nonce":3479894866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3479894866},{"type":"v1","addr":"172.21.15.120:6809","nonce":3479894866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3479894866},{"type":"v1","addr":"172.21.15.120:6807","nonce":3479894866}]},"public_addr":"172.21.15.120:6803/3479894866","cluster_addr":"172.21.15.120:6805/3479894866","heartbeat_back_addr":"172.21.15.120:6809/3479894866","heartbeat_front_addr":"172.21.15.120:6807/3479894866","state":["exists","up"]},{"osd":1,"uuid":"58240258-afd5-4991-9d5b-a80d365e4fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3046662876},{"type":"v1","addr":"172.21.15.134:6801","nonce":3046662876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3046662876},{"type":"v1","addr":"172.21.15.134:6803","nonce":3046662876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3046662876},{"type":"v1","addr":"172.21.15.134:6807","nonce":3046662876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3046662876},{"type":"v1","addr":"172.21.15.134:6805","nonce":3046662876}]},"public_addr":"172.21.15.134:6801/3046662876","cluster_addr":"172.21.15.134:6803/3046662876","heartbeat_back_addr":"172.21.15.134:6807/3046662876","heartbeat_front_addr":"172.21.15.134:6805/3046662876","state":["exists","up"]},{"osd":2,"uuid":"05988a7a-7cf2-4fbb-9d4f-22604cd2f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3996265223},{"type":"v1","addr":"172.21.15.157:6801","nonce":3996265223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3996265223},{"type":"v1","addr":"172.21.15.157:6803","nonce":3996265223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3996265223},{"type":"v1","addr":"172.21.15.157:6807","nonce":3996265223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3996265223},{"type":"v1","addr":"172.21.15.157:6805","nonce":3996265223}]},"public_addr":"172.21.15.157:6801/3996265223","cluster_addr":"172.21.15.157:6803/3996265223","heartbeat_back_addr":"172.21.15.157:6807/3996265223","heartbeat_front_addr":"172.21.15.157:6805/3996265223","state":["exists","up"]},{"osd":3,"uuid":"7d9e9739-f043-43d6-8e46-7711dcf43e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":1716638437},{"type":"v1","addr":"172.21.15.120:6811","nonce":1716638437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":1716638437},{"type":"v1","addr":"172.21.15.120:6813","nonce":1716638437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1716638437},{"type":"v1","addr":"172.21.15.120:6817","nonce":1716638437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":1716638437},{"type":"v1","addr":"172.21.15.120:6815","nonce":1716638437}]},"public_addr":"172.21.15.120:6811/1716638437","cluster_addr":"172.21.15.120:6813/1716638437","heartbeat_back_addr":"172.21.15.120:6817/1716638437","heartbeat_front_addr":"172.21.15.120:6815/1716638437","state":["exists","up"]},{"osd":4,"uuid":"3a70f86d-f903-4618-be66-9aa1eac44ccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4172238262},{"type":"v1","addr":"172.21.15.134:6809","nonce":4172238262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4172238262},{"type":"v1","addr":"172.21.15.134:6811","nonce":4172238262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4172238262},{"type":"v1","addr":"172.21.15.134:6815","nonce":4172238262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4172238262},{"type":"v1","addr":"172.21.15.134:6813","nonce":4172238262}]},"public_addr":"172.21.15.134:6809/4172238262","cluster_addr":"172.21.15.134:6811/4172238262","heartbeat_back_addr":"172.21.15.134:6815/4172238262","heartbeat_front_addr":"172.21.15.134:6813/4172238262","state":["exists","up"]},{"osd":5,"uuid":"b2316488-75c3-4847-b4c4-998730f27c88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3374822045},{"type":"v1","addr":"172.21.15.157:6809","nonce":3374822045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3374822045},{"type":"v1","addr":"172.21.15.157:6811","nonce":3374822045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3374822045},{"type":"v1","addr":"172.21.15.157:6815","nonce":3374822045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3374822045},{"type":"v1","addr":"172.21.15.157:6813","nonce":3374822045}]},"public_addr":"172.21.15.157:6809/3374822045","cluster_addr":"172.21.15.157:6811/3374822045","heartbeat_back_addr":"172.21.15.157:6815/3374822045","heartbeat_front_addr":"172.21.15.157:6813/3374822045","state":["exists","up"]},{"osd":6,"uuid":"338c2c4a-d982-4031-aaaf-be76c9dd74a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2097345225},{"type":"v1","addr":"172.21.15.120:6819","nonce":2097345225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2097345225},{"type":"v1","addr":"172.21.15.120:6821","nonce":2097345225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2097345225},{"type":"v1","addr":"172.21.15.120:6825","nonce":2097345225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2097345225},{"type":"v1","addr":"172.21.15.120:6823","nonce":2097345225}]},"public_addr":"172.21.15.120:6819/2097345225","cluster_addr":"172.21.15.120:6821/2097345225","heartbeat_back_addr":"172.21.15.120:6825/2097345225","heartbeat_front_addr":"172.21.15.120:6823/2097345225","state":["exists","up"]},{"osd":7,"uuid":"bff92799-ae9c-4409-be65-7ffc27a1c3f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1345745298},{"type":"v1","addr":"172.21.15.134:6817","nonce":1345745298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1345745298},{"type":"v1","addr":"172.21.15.134:6819","nonce":1345745298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1345745298},{"type":"v1","addr":"172.21.15.134:6823","nonce":1345745298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1345745298},{"type":"v1","addr":"172.21.15.134:6821","nonce":1345745298}]},"public_addr":"172.21.15.134:6817/1345745298","cluster_addr":"172.21.15.134:6819/1345745298","heartbeat_back_addr":"172.21.15.134:6823/1345745298","heartbeat_front_addr":"172.21.15.134:6821/1345745298","state":["exists","up"]},{"osd":8,"uuid":"e4aac47e-d7f9-4feb-ada7-d8103fdca246","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1124549599},{"type":"v1","addr":"172.21.15.157:6817","nonce":1124549599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1124549599},{"type":"v1","addr":"172.21.15.157:6819","nonce":1124549599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1124549599},{"type":"v1","addr":"172.21.15.157:6823","nonce":1124549599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1124549599},{"type":"v1","addr":"172.21.15.157:6821","nonce":1124549599}]},"public_addr":"172.21.15.157:6817/1124549599","cluster_addr":"172.21.15.157:6819/1124549599","heartbeat_back_addr":"172.21.15.157:6823/1124549599","heartbeat_front_addr":"172.21.15.157:6821/1124549599","state":["exists","up"]},{"osd":9,"uuid":"48b988ba-e866-47d1-aca9-09f6ad815bd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1402414981},{"type":"v1","addr":"172.21.15.120:6827","nonce":1402414981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1402414981},{"type":"v1","addr":"172.21.15.120:6829","nonce":1402414981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1402414981},{"type":"v1","addr":"172.21.15.120:6833","nonce":1402414981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1402414981},{"type":"v1","addr":"172.21.15.120:6831","nonce":1402414981}]},"public_addr":"172.21.15.120:6827/1402414981","cluster_addr":"172.21.15.120:6829/1402414981","heartbeat_back_addr":"172.21.15.120:6833/1402414981","heartbeat_front_addr":"172.21.15.120:6831/1402414981","state":["exists","up"]},{"osd":10,"uuid":"777f3add-3779-4221-a86f-65cca2e5e8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":4041441144},{"type":"v1","addr":"172.21.15.134:6825","nonce":4041441144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":4041441144},{"type":"v1","addr":"172.21.15.134:6827","nonce":4041441144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":4041441144},{"type":"v1","addr":"172.21.15.134:6831","nonce":4041441144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":4041441144},{"type":"v1","addr":"172.21.15.134:6829","nonce":4041441144}]},"public_addr":"172.21.15.134:6825/4041441144","cluster_addr":"172.21.15.134:6827/4041441144","heartbeat_back_addr":"172.21.15.134:6831/4041441144","heartbeat_front_addr":"172.21.15.134:6829/4041441144","state":["exists","up"]},{"osd":11,"uuid":"7166e11b-03c0-45bf-abbf-a74c4a13a39d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1537477260},{"type":"v1","addr":"172.21.15.157:6825","nonce":1537477260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1537477260},{"type":"v1","addr":"172.21.15.157:6827","nonce":1537477260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1537477260},{"type":"v1","addr":"172.21.15.157:6831","nonce":1537477260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1537477260},{"type":"v1","addr":"172.21.15.157:6829","nonce":1537477260}]},"public_addr":"172.21.15.157:6825/1537477260","cluster_addr":"172.21.15.157:6827/1537477260","heartbeat_back_addr":"172.21.15.157:6831/1537477260","heartbeat_front_addr":"172.21.15.157:6829/1537477260","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:13:13.580631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:13:37.472924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:14:01.591555+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:14:28.472130+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:14:52.356433+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:15:15.946523+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:15:42.036105+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:16:06.154934+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:16:30.043788+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:16:55.975403+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:17:20.612778+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:17:44.896301+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.120:0/2865552846":"2023-06-24T01:11:38.967204+0000","172.21.15.120:0/2163781512":"2023-06-24T01:11:38.967204+0000","172.21.15.120:0/1943345772":"2023-06-24T01:11:38.967204+0000","172.21.15.120:6801/1836713308":"2023-06-24T01:11:38.967204+0000","172.21.15.120:0/2358086231":"2023-06-24T01:11:14.574788+0000","172.21.15.120:6801/506843621":"2023-06-24T01:11:14.574788+0000","172.21.15.120:6800/506843621":"2023-06-24T01:11:14.574788+0000","172.21.15.120:0/4077038498":"2023-06-24T01:11:14.574788+0000","172.21.15.120:6800/1836713308":"2023-06-24T01:11:38.967204+0000","172.21.15.120:0/2484415475":"2023-06-24T01:11:14.574788+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T01:18:38.501 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.497+0000 7f19f5b9d700 1 -- 172.21.15.120:0/2416356178 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f19e009b680 msgr2=0x7f19e009db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:38.501 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.497+0000 7f19f5b9d700 1 --2- 172.21.15.120:0/2416356178 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f19e009b680 0x7f19e009db30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f19dc004890 tx=0x7f19dc009e30 comp rx=0 tx=0).stop 2023-06-23T01:18:38.501 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.497+0000 7f19f5b9d700 1 -- 172.21.15.120:0/2416356178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f0065c10 msgr2=0x7f19f00fcfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:38.502 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.497+0000 7f19f5b9d700 1 --2- 172.21.15.120:0/2416356178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f0065c10 0x7f19f00fcfd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f19e400ebf0 tx=0x7f19e400b040 comp rx=0 tx=0).stop 2023-06-23T01:18:38.502 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.497+0000 7f19f5b9d700 1 -- 172.21.15.120:0/2416356178 shutdown_connections 2023-06-23T01:18:38.502 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.497+0000 7f19f5b9d700 1 --2- 172.21.15.120:0/2416356178 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f19e009b680 0x7f19e009db30 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:38.502 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.497+0000 7f19f5b9d700 1 --2- 172.21.15.120:0/2416356178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19f0102760 0x7f19f00fbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:38.502 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.497+0000 7f19f5b9d700 1 --2- 172.21.15.120:0/2416356178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f0065c10 0x7f19f00fcfd0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:38.503 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.497+0000 7f19f5b9d700 1 --2- 172.21.15.120:0/2416356178 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19f00ffc70 0x7f19f00fb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:38.503 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.497+0000 7f19f5b9d700 1 -- 172.21.15.120:0/2416356178 >> 172.21.15.120:0/2416356178 conn(0x7f19f00f7140 msgr2=0x7f19f00fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:38.503 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.497+0000 7f19f5b9d700 1 -- 172.21.15.120:0/2416356178 shutdown_connections 2023-06-23T01:18:38.503 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.497+0000 7f19f5b9d700 1 -- 172.21.15.120:0/2416356178 wait complete. 2023-06-23T01:18:38.510 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:38 smithi120 ceph-mon[79813]: pgmap v281: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T01:18:38.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:38 smithi120 ceph-mon[79813]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-06-23T01:18:38.511 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:38 smithi120 ceph-mon[79813]: osdmap e83: 12 total, 12 up, 12 in 2023-06-23T01:18:38.512 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:38 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/1138423833' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-06-23T01:18:38.512 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:38 smithi120 ceph-mon[79813]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-06-23T01:18:38.512 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:38 smithi120 ceph-mon[79813]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-06-23T01:18:38.513 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:38 smithi120 ceph-mon[79813]: osdmap e84: 12 total, 12 up, 12 in 2023-06-23T01:18:38.513 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:38 smithi120 ceph-mon[79813]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-06-23T01:18:38.513 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:38 smithi120 ceph-mon[79813]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-06-23T01:18:38.513 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:38 smithi120 ceph-mon[79813]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-06-23T01:18:38.514 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:38 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/584053984' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T01:18:38.519 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-06-23T01:18:38.721 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.717+0000 7f8e5a79d700 1 -- 172.21.15.120:0/1997003586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e54065c70 msgr2=0x7f8e540ff6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:38.721 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.717+0000 7f8e5a79d700 1 --2- 172.21.15.120:0/1997003586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e54065c70 0x7f8e540ff6b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8e500049c0 tx=0x7f8e50009e30 comp rx=0 tx=0).stop 2023-06-23T01:18:38.721 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.717+0000 7f8e5a79d700 1 -- 172.21.15.120:0/1997003586 shutdown_connections 2023-06-23T01:18:38.722 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.717+0000 7f8e5a79d700 1 --2- 172.21.15.120:0/1997003586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e54102740 0x7f8e54104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:38.722 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.717+0000 7f8e5a79d700 1 --2- 172.21.15.120:0/1997003586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e54065c70 0x7f8e540ff6b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:38.722 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.717+0000 7f8e5a79d700 1 --2- 172.21.15.120:0/1997003586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8e540ffbf0 0x7f8e54102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:38.722 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.717+0000 7f8e5a79d700 1 -- 172.21.15.120:0/1997003586 >> 172.21.15.120:0/1997003586 conn(0x7f8e540f7100 msgr2=0x7f8e540f9550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:38.722 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.717+0000 7f8e5a79d700 1 -- 172.21.15.120:0/1997003586 shutdown_connections 2023-06-23T01:18:38.723 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.717+0000 7f8e5a79d700 1 -- 172.21.15.120:0/1997003586 wait complete. 2023-06-23T01:18:38.724 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.718+0000 7f8e5a79d700 1 Processor -- start 2023-06-23T01:18:38.724 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.718+0000 7f8e5a79d700 1 -- start start 2023-06-23T01:18:38.724 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.719+0000 7f8e5a79d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e54065c70 0x7f8e540fcf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:38.725 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.719+0000 7f8e5a79d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8e540ffbf0 0x7f8e540fb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:38.725 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.719+0000 7f8e5a79d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e54102740 0x7f8e540fbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:38.725 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.719+0000 7f8e5a79d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e540fc260 con 0x7f8e540ffbf0 2023-06-23T01:18:38.725 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.719+0000 7f8e5a79d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e540fc3d0 con 0x7f8e54102740 2023-06-23T01:18:38.725 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.719+0000 7f8e5a79d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e541178a0 con 0x7f8e54065c70 2023-06-23T01:18:38.725 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.719+0000 7f8e58f9a700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8e540ffbf0 0x7f8e540fb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:38.725 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.719+0000 7f8e58f9a700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8e540ffbf0 0x7f8e540fb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38728/0 (socket says 172.21.15.120:38728) 2023-06-23T01:18:38.725 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.719+0000 7f8e58f9a700 1 -- 172.21.15.120:0/3249048502 learned_addr learned my addr 172.21.15.120:0/3249048502 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:18:38.726 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.719+0000 7f8e59f9c700 1 --2- 172.21.15.120:0/3249048502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e54102740 0x7f8e540fbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:38.726 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.719+0000 7f8e5979b700 1 --2- 172.21.15.120:0/3249048502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e54065c70 0x7f8e540fcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:38.726 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.720+0000 7f8e59f9c700 1 -- 172.21.15.120:0/3249048502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e54065c70 msgr2=0x7f8e540fcf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:38.726 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.720+0000 7f8e59f9c700 1 --2- 172.21.15.120:0/3249048502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e54065c70 0x7f8e540fcf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:38.726 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.720+0000 7f8e59f9c700 1 -- 172.21.15.120:0/3249048502 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8e540ffbf0 msgr2=0x7f8e540fb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:38.726 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.720+0000 7f8e59f9c700 1 --2- 172.21.15.120:0/3249048502 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8e540ffbf0 0x7f8e540fb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:38.726 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.720+0000 7f8e59f9c700 1 -- 172.21.15.120:0/3249048502 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e50005040 con 0x7f8e54102740 2023-06-23T01:18:38.727 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.720+0000 7f8e59f9c700 1 --2- 172.21.15.120:0/3249048502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e54102740 0x7f8e540fbaf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c00ec20 tx=0x7f8e4c00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:38.727 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.720+0000 7f8e49ffb700 1 -- 172.21.15.120:0/3249048502 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c007b30 con 0x7f8e54102740 2023-06-23T01:18:38.727 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.720+0000 7f8e5a79d700 1 -- 172.21.15.120:0/3249048502 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e54117b50 con 0x7f8e54102740 2023-06-23T01:18:38.727 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.720+0000 7f8e49ffb700 1 -- 172.21.15.120:0/3249048502 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e4c00f040 con 0x7f8e54102740 2023-06-23T01:18:38.727 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.720+0000 7f8e49ffb700 1 -- 172.21.15.120:0/3249048502 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c00c9f0 con 0x7f8e54102740 2023-06-23T01:18:38.727 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.720+0000 7f8e5a79d700 1 -- 172.21.15.120:0/3249048502 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e54118040 con 0x7f8e54102740 2023-06-23T01:18:38.728 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.722+0000 7f8e5a79d700 1 -- 172.21.15.120:0/3249048502 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e540f8c30 con 0x7f8e54102740 2023-06-23T01:18:38.729 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.726+0000 7f8e49ffb700 1 -- 172.21.15.120:0/3249048502 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7f8e4c009090 con 0x7f8e54102740 2023-06-23T01:18:38.730 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.727+0000 7f8e49ffb700 1 --2- 172.21.15.120:0/3249048502 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f8e4009b700 0x7f8e4009dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:38.730 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.727+0000 7f8e49ffb700 1 -- 172.21.15.120:0/3249048502 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7975+0+0 (secure 0 0 0) 0x7f8e4c0c5610 con 0x7f8e54102740 2023-06-23T01:18:38.731 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.727+0000 7f8e5979b700 1 --2- 172.21.15.120:0/3249048502 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f8e4009b700 0x7f8e4009dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:38.731 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.728+0000 7f8e49ffb700 1 -- 172.21.15.120:0/3249048502 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f8e4c0949b0 con 0x7f8e54102740 2023-06-23T01:18:38.731 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:38.728+0000 7f8e5979b700 1 --2- 172.21.15.120:0/3249048502 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f8e4009b700 0x7f8e4009dbb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8e50005010 tx=0x7f8e50009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:38.995 INFO:teuthology.orchestra.run.smithi120.stderr:no valid command found; 1 closest matches: 2023-06-23T01:18:38.996 INFO:teuthology.orchestra.run.smithi120.stderr:fs set |refuse_client_session|max_xattr_size [--yes-i-really-mean-it] [--yes-i-really-really-mean-it] 2023-06-23T01:18:38.996 INFO:teuthology.orchestra.run.smithi120.stderr:Error EINVAL: invalid command 2023-06-23T01:18:39.003 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-06-23T01:18:39.004 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2023-06-23T01:18:39.004 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2023-06-23T01:18:39.201 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.197+0000 7f37c219d700 1 -- 172.21.15.120:0/2940653007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37bc0fe680 msgr2=0x7f37bc104f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:39.201 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.197+0000 7f37c219d700 1 --2- 172.21.15.120:0/2940653007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37bc0fe680 0x7f37bc104f40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f37b4009d60 tx=0x7f37b4004d10 comp rx=0 tx=0).stop 2023-06-23T01:18:39.201 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.197+0000 7f37c219d700 1 -- 172.21.15.120:0/2940653007 shutdown_connections 2023-06-23T01:18:39.201 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.197+0000 7f37c219d700 1 --2- 172.21.15.120:0/2940653007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37bc0fe680 0x7f37bc104f40 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:39.201 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.197+0000 7f37c219d700 1 --2- 172.21.15.120:0/2940653007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37bc0fcae0 0x7f37bc0fcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:39.201 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.197+0000 7f37c219d700 1 --2- 172.21.15.120:0/2940653007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37bc0fdcf0 0x7f37bc0fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:39.202 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.197+0000 7f37c219d700 1 -- 172.21.15.120:0/2940653007 >> 172.21.15.120:0/2940653007 conn(0x7f37bc0f8200 msgr2=0x7f37bc0fa670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:18:39.202 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.197+0000 7f37c219d700 1 -- 172.21.15.120:0/2940653007 shutdown_connections 2023-06-23T01:18:39.202 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.198+0000 7f37c219d700 1 -- 172.21.15.120:0/2940653007 wait complete. 2023-06-23T01:18:39.204 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.198+0000 7f37c219d700 1 Processor -- start 2023-06-23T01:18:39.204 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.198+0000 7f37c219d700 1 -- start start 2023-06-23T01:18:39.204 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.199+0000 7f37c219d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37bc0fcae0 0x7f37bc071720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:39.204 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.199+0000 7f37c219d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37bc0fdcf0 0x7f37bc06fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:39.204 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.199+0000 7f37c219d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37bc0fe680 0x7f37bc0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:39.205 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.199+0000 7f37c219d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc070a30 con 0x7f37bc0fdcf0 2023-06-23T01:18:39.205 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.199+0000 7f37c219d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc070ba0 con 0x7f37bc0fe680 2023-06-23T01:18:39.205 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.199+0000 7f37c219d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc117720 con 0x7f37bc0fcae0 2023-06-23T01:18:39.205 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.199+0000 7f37c119b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37bc0fcae0 0x7f37bc071720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:39.206 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.199+0000 7f37c119b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37bc0fcae0 0x7f37bc071720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.120:49536/0 (socket says 172.21.15.120:49536) 2023-06-23T01:18:39.206 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.199+0000 7f37c119b700 1 -- 172.21.15.120:0/3711588643 learned_addr learned my addr 172.21.15.120:0/3711588643 (peer_addr_for_me v2:172.21.15.120:0/0) 2023-06-23T01:18:39.206 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.199+0000 7f37c099a700 1 --2- 172.21.15.120:0/3711588643 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37bc0fdcf0 0x7f37bc06fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:39.206 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.199+0000 7f37c199c700 1 --2- 172.21.15.120:0/3711588643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37bc0fe680 0x7f37bc0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:39.206 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.199+0000 7f37c119b700 1 -- 172.21.15.120:0/3711588643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37bc0fe680 msgr2=0x7f37bc0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:39.206 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.199+0000 7f37c119b700 1 --2- 172.21.15.120:0/3711588643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37bc0fe680 0x7f37bc0702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:39.207 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.199+0000 7f37c119b700 1 -- 172.21.15.120:0/3711588643 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37bc0fdcf0 msgr2=0x7f37bc06fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:18:39.207 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.199+0000 7f37c119b700 1 --2- 172.21.15.120:0/3711588643 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37bc0fdcf0 0x7f37bc06fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:18:39.207 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.199+0000 7f37c119b700 1 -- 172.21.15.120:0/3711588643 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b4009940 con 0x7f37bc0fcae0 2023-06-23T01:18:39.207 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.199+0000 7f37c119b700 1 --2- 172.21.15.120:0/3711588643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37bc0fcae0 0x7f37bc071720 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f37b800ebf0 tx=0x7f37b800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:39.207 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.199+0000 7f37b1ffb700 1 -- 172.21.15.120:0/3711588643 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b8007cc0 con 0x7f37bc0fcae0 2023-06-23T01:18:39.207 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.200+0000 7f37c219d700 1 -- 172.21.15.120:0/3711588643 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37bc117970 con 0x7f37bc0fcae0 2023-06-23T01:18:39.207 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.200+0000 7f37b1ffb700 1 -- 172.21.15.120:0/3711588643 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37b800f040 con 0x7f37bc0fcae0 2023-06-23T01:18:39.207 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.200+0000 7f37b1ffb700 1 -- 172.21.15.120:0/3711588643 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b800c8f0 con 0x7f37bc0fcae0 2023-06-23T01:18:39.208 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.200+0000 7f37c219d700 1 -- 172.21.15.120:0/3711588643 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37bc117ee0 con 0x7f37bc0fcae0 2023-06-23T01:18:39.208 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.201+0000 7f37c219d700 1 -- 172.21.15.120:0/3711588643 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37bc103e80 con 0x7f37bc0fcae0 2023-06-23T01:18:39.208 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.202+0000 7f37b1ffb700 1 -- 172.21.15.120:0/3711588643 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 129202+0+0 (secure 0 0 0) 0x7f37b800cd20 con 0x7f37bc0fcae0 2023-06-23T01:18:39.209 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.204+0000 7f37b1ffb700 1 --2- 172.21.15.120:0/3711588643 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f37a809b680 0x7f37a809db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:18:39.209 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.204+0000 7f37b1ffb700 1 -- 172.21.15.120:0/3711588643 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7975+0+0 (secure 0 0 0) 0x7f37b80c4760 con 0x7f37bc0fcae0 2023-06-23T01:18:39.209 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.204+0000 7f37c099a700 1 --2- 172.21.15.120:0/3711588643 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f37a809b680 0x7f37a809db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:18:39.209 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.204+0000 7f37c099a700 1 --2- 172.21.15.120:0/3711588643 >> [v2:172.21.15.120:6800/1770912857,v1:172.21.15.120:6801/1770912857] conn(0x7f37a809b680 0x7f37a809db30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f37ac009b80 tx=0x7f37ac00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:18:39.210 INFO:teuthology.orchestra.run.smithi120.stderr:2023-06-23T01:18:39.207+0000 7f37b1ffb700 1 -- 172.21.15.120:0/3711588643 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f37b804da00 con 0x7f37bc0fcae0 2023-06-23T01:18:39.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:39 smithi120 ceph-mon[79813]: from='client.? 172.21.15.120:0/2416356178' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T01:18:39.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:39 smithi120 ceph-mon[79813]: osdmap e85: 12 total, 12 up, 12 in 2023-06-23T01:18:39.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:39 smithi120 ceph-mon[79813]: mds.? [v2:172.21.15.134:6838/1531909538,v1:172.21.15.134:6839/1531909538] up:active 2023-06-23T01:18:39.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:39 smithi120 ceph-mon[79813]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-06-23T01:18:39.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:39 smithi120 ceph-mon[79813]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-06-23T01:18:39.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:39 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:39.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:39 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:39.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:39 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:39.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:39 smithi120 ceph-mon[79813]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:39.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:39 smithi157 ceph-mon[116945]: from='client.? 172.21.15.120:0/2416356178' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T01:18:39.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:39 smithi157 ceph-mon[116945]: osdmap e85: 12 total, 12 up, 12 in 2023-06-23T01:18:39.377 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:39 smithi157 ceph-mon[116945]: mds.? [v2:172.21.15.134:6838/1531909538,v1:172.21.15.134:6839/1531909538] up:active 2023-06-23T01:18:39.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:39 smithi157 ceph-mon[116945]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-06-23T01:18:39.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:39 smithi157 ceph-mon[116945]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-06-23T01:18:39.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:39 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:39.378 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:39 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:39.379 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:39 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:39.379 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:39 smithi157 ceph-mon[116945]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:39.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:39 smithi134 ceph-mon[116146]: from='client.? 172.21.15.120:0/2416356178' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T01:18:39.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:39 smithi134 ceph-mon[116146]: osdmap e85: 12 total, 12 up, 12 in 2023-06-23T01:18:39.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:39 smithi134 ceph-mon[116146]: mds.? [v2:172.21.15.134:6838/1531909538,v1:172.21.15.134:6839/1531909538] up:active 2023-06-23T01:18:39.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:39 smithi134 ceph-mon[116146]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-06-23T01:18:39.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:39 smithi134 ceph-mon[116146]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-06-23T01:18:39.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:39 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:39.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:39 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:39.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:39 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:39.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:39 smithi134 ceph-mon[116146]: from='mgr.14152 172.21.15.120:0/807722839' entity='mgr.x' 2023-06-23T01:18:39.463 INFO:teuthology.orchestra.run.smithi120.stderr:no valid command found; 1 closest matches: 2023-06-23T01:18:39.463 INFO:teuthology.orchestra.run.smithi120.stderr:fs set |refuse_client_session|max_xattr_size [--yes-i-really-mean-it] [--yes-i-really-really-mean-it] 2023-06-23T01:18:39.463 INFO:teuthology.orchestra.run.smithi120.stderr:Error EINVAL: invalid command 2023-06-23T01:18:39.471 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-06-23T01:18:39.472 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephadm.py", line 896, in cephfs_setup fs = Filesystem(ctx, fs_config=temp, name=name, create=True) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 514, in __init__ self.create() File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 714, in create self.set_max_mds(max_mds) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 583, in set_max_mds self.set_var("max_mds", "%d" % max_mds) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 574, in set_var self.mon_manager.raw_cluster_cmd("fs", "set", self.name, var, *a) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph_manager.py", line 1624, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph_manager.py", line 1615, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi120 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5' 2023-06-23T01:18:39.473 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-06-23T01:18:39.537 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-06-23T01:18:39.572 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-06-23T01:18:39.604 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-06-23T01:18:39.605 DEBUG:teuthology.orchestra.run.smithi120:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-06-23T01:18:39.641 DEBUG:teuthology.orchestra.run.smithi134:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-06-23T01:18:39.659 DEBUG:teuthology.orchestra.run.smithi157:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-06-23T01:18:39.676 INFO:tasks.cephadm:Stopping all daemons... 2023-06-23T01:18:39.677 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-06-23T01:18:39.677 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mon.a 2023-06-23T01:18:40.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:39 smithi120 systemd[1]: Stopping Ceph mon.a for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:18:40.747 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:40 smithi120 conmon[79789]: 2023-06-23T01:18:40.471+0000 7fae8cb49700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T01:18:40.747 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:40 smithi120 conmon[79789]: 2023-06-23T01:18:40.471+0000 7fae8cb49700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-06-23T01:18:41.010 INFO:journalctl@ceph.mon.a.smithi120.stdout:Jun 23 01:18:40 smithi120 bash[126805]: ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c-mon-a 2023-06-23T01:18:41.270 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mon.a.service' 2023-06-23T01:18:41.360 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T01:18:41.361 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-06-23T01:18:41.361 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-06-23T01:18:41.362 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mon.b 2023-06-23T01:18:41.694 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:41 smithi134 systemd[1]: Stopping Ceph mon.b for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:18:42.095 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:41 smithi134 conmon[116121]: 2023-06-23T01:18:41.835+0000 7f368dc60700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T01:18:42.096 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:41 smithi134 conmon[116121]: 2023-06-23T01:18:41.835+0000 7f368dc60700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-06-23T01:18:42.096 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jun 23 01:18:42 smithi134 bash[150855]: ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c-mon-b 2023-06-23T01:18:42.920 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mon.b.service' 2023-06-23T01:18:42.965 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T01:18:42.965 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-06-23T01:18:42.966 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-06-23T01:18:42.966 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mon.c 2023-06-23T01:18:43.194 INFO:journalctl@ceph.mds.b.smithi134.stdout:Jun 23 01:18:43 smithi134 conmon[148932]: 2023-06-23T01:18:43.004+0000 7f92c2145700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-06-23T01:18:43.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:42 smithi157 systemd[1]: Stopping Ceph mon.c for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:18:43.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:43 smithi157 conmon[116894]: 2023-06-23T01:18:43.388+0000 7f02dbdf5700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T01:18:43.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:43 smithi157 conmon[116894]: 2023-06-23T01:18:43.388+0000 7f02dbdf5700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-06-23T01:18:44.659 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:44 smithi157 bash[153325]: ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c-mon-c 2023-06-23T01:18:44.659 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:44 smithi157 systemd[1]: ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mon.c.service: Succeeded. 2023-06-23T01:18:44.659 INFO:journalctl@ceph.mon.c.smithi157.stdout:Jun 23 01:18:44 smithi157 systemd[1]: Stopped Ceph mon.c for 9efbfa8e-1162-11ee-9b2b-001a4aab830c. 2023-06-23T01:18:44.661 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mon.c.service' 2023-06-23T01:18:44.743 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T01:18:44.744 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-06-23T01:18:44.744 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-06-23T01:18:44.745 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mgr.x 2023-06-23T01:18:45.260 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:18:44 smithi120 systemd[1]: Stopping Ceph mgr.x for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:18:45.594 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:18:45 smithi120 conmon[80224]: 2023-06-23T01:18:45.337+0000 7f391a7b1700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T01:18:45.595 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Jun 23 01:18:45 smithi120 bash[127815]: ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c-mgr-x 2023-06-23T01:18:45.987 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mgr.x.service' 2023-06-23T01:18:46.070 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T01:18:46.070 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-06-23T01:18:46.071 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-06-23T01:18:46.071 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mgr.y 2023-06-23T01:18:46.444 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jun 23 01:18:46 smithi134 systemd[1]: Stopping Ceph mgr.y for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:18:47.218 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mgr.y.service' 2023-06-23T01:18:47.241 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jun 23 01:18:46 smithi134 bash[151534]: ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c-mgr-y 2023-06-23T01:18:47.241 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jun 23 01:18:46 smithi134 systemd[1]: ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-06-23T01:18:47.242 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jun 23 01:18:47 smithi134 systemd[1]: ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mgr.y.service: Failed with result 'exit-code'. 2023-06-23T01:18:47.242 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jun 23 01:18:47 smithi134 systemd[1]: Stopped Ceph mgr.y for 9efbfa8e-1162-11ee-9b2b-001a4aab830c. 2023-06-23T01:18:47.255 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T01:18:47.255 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-06-23T01:18:47.256 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-06-23T01:18:47.256 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mgr.z 2023-06-23T01:18:47.626 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Jun 23 01:18:47 smithi157 systemd[1]: Stopping Ceph mgr.z for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:18:48.399 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mgr.z.service' 2023-06-23T01:18:48.434 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T01:18:48.435 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-06-23T01:18:48.436 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-06-23T01:18:48.437 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@osd.0 2023-06-23T01:18:48.760 INFO:journalctl@ceph.osd.0.smithi120.stdout:Jun 23 01:18:48 smithi120 systemd[1]: Stopping Ceph osd.0 for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:18:49.260 INFO:journalctl@ceph.osd.0.smithi120.stdout:Jun 23 01:18:48 smithi120 conmon[96119]: 2023-06-23T01:18:48.944+0000 7fdc2c8c0700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T01:18:49.260 INFO:journalctl@ceph.osd.0.smithi120.stdout:Jun 23 01:18:48 smithi120 conmon[96119]: 2023-06-23T01:18:48.944+0000 7fdc2c8c0700 -1 osd.0 85 *** Got signal Terminated *** 2023-06-23T01:18:49.261 INFO:journalctl@ceph.osd.0.smithi120.stdout:Jun 23 01:18:48 smithi120 conmon[96119]: 2023-06-23T01:18:48.944+0000 7fdc2c8c0700 -1 osd.0 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-23T01:18:54.760 INFO:journalctl@ceph.osd.0.smithi120.stdout:Jun 23 01:18:54 smithi120 bash[128101]: ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c-osd-0 2023-06-23T01:18:56.390 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@osd.0.service' 2023-06-23T01:18:56.472 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T01:18:56.473 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-06-23T01:18:56.473 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-06-23T01:18:56.473 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@osd.1 2023-06-23T01:18:56.824 INFO:journalctl@ceph.osd.1.smithi134.stdout:Jun 23 01:18:56 smithi134 systemd[1]: Stopping Ceph osd.1 for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:18:57.193 INFO:journalctl@ceph.osd.1.smithi134.stdout:Jun 23 01:18:56 smithi134 conmon[122645]: 2023-06-23T01:18:56.820+0000 7fc894815700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T01:18:57.194 INFO:journalctl@ceph.osd.1.smithi134.stdout:Jun 23 01:18:56 smithi134 conmon[122645]: 2023-06-23T01:18:56.820+0000 7fc894815700 -1 osd.1 85 *** Got signal Terminated *** 2023-06-23T01:18:57.194 INFO:journalctl@ceph.osd.1.smithi134.stdout:Jun 23 01:18:56 smithi134 conmon[122645]: 2023-06-23T01:18:56.820+0000 7fc894815700 -1 osd.1 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-23T01:19:02.944 INFO:journalctl@ceph.osd.1.smithi134.stdout:Jun 23 01:19:02 smithi134 bash[151815]: ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c-osd-1 2023-06-23T01:19:04.476 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@osd.1.service' 2023-06-23T01:19:04.513 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T01:19:04.514 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-06-23T01:19:04.515 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-06-23T01:19:04.515 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@osd.2 2023-06-23T01:19:04.863 INFO:journalctl@ceph.osd.2.smithi157.stdout:Jun 23 01:19:04 smithi157 systemd[1]: Stopping Ceph osd.2 for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:19:05.126 INFO:journalctl@ceph.osd.2.smithi157.stdout:Jun 23 01:19:04 smithi157 conmon[124323]: 2023-06-23T01:19:04.859+0000 7fe648a4c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T01:19:05.126 INFO:journalctl@ceph.osd.2.smithi157.stdout:Jun 23 01:19:04 smithi157 conmon[124323]: 2023-06-23T01:19:04.859+0000 7fe648a4c700 -1 osd.2 85 *** Got signal Terminated *** 2023-06-23T01:19:05.127 INFO:journalctl@ceph.osd.2.smithi157.stdout:Jun 23 01:19:04 smithi157 conmon[124323]: 2023-06-23T01:19:04.859+0000 7fe648a4c700 -1 osd.2 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-23T01:19:10.876 INFO:journalctl@ceph.osd.2.smithi157.stdout:Jun 23 01:19:10 smithi157 bash[153892]: ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c-osd-2 2023-06-23T01:19:12.554 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@osd.2.service' 2023-06-23T01:19:12.592 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T01:19:12.593 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-06-23T01:19:12.594 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-06-23T01:19:12.594 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@osd.3 2023-06-23T01:19:13.010 INFO:journalctl@ceph.osd.3.smithi120.stdout:Jun 23 01:19:12 smithi120 systemd[1]: Stopping Ceph osd.3 for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:19:13.510 INFO:journalctl@ceph.osd.3.smithi120.stdout:Jun 23 01:19:13 smithi120 conmon[103072]: 2023-06-23T01:19:13.099+0000 7fd24b516700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T01:19:13.510 INFO:journalctl@ceph.osd.3.smithi120.stdout:Jun 23 01:19:13 smithi120 conmon[103072]: 2023-06-23T01:19:13.099+0000 7fd24b516700 -1 osd.3 85 *** Got signal Terminated *** 2023-06-23T01:19:13.511 INFO:journalctl@ceph.osd.3.smithi120.stdout:Jun 23 01:19:13 smithi120 conmon[103072]: 2023-06-23T01:19:13.100+0000 7fd24b516700 -1 osd.3 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-23T01:19:14.876 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:14 smithi157 conmon[145357]: 2023-06-23T01:19:14.460+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:15.010 INFO:journalctl@ceph.osd.3.smithi120.stdout:Jun 23 01:19:14 smithi120 conmon[103072]: 2023-06-23T01:19:14.717+0000 7fd246b12700 -1 osd.3 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.249541+0000 front 2023-06-23T01:18:50.249572+0000 (oldest deadline 2023-06-23T01:19:14.349769+0000) 2023-06-23T01:19:15.876 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:15 smithi157 conmon[145357]: 2023-06-23T01:19:15.458+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:16.010 INFO:journalctl@ceph.osd.3.smithi120.stdout:Jun 23 01:19:15 smithi120 conmon[103072]: 2023-06-23T01:19:15.743+0000 7fd246b12700 -1 osd.3 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.249541+0000 front 2023-06-23T01:18:50.249572+0000 (oldest deadline 2023-06-23T01:19:14.349769+0000) 2023-06-23T01:19:16.876 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:16 smithi157 conmon[145357]: 2023-06-23T01:19:16.474+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:17.260 INFO:journalctl@ceph.osd.3.smithi120.stdout:Jun 23 01:19:16 smithi120 conmon[103072]: 2023-06-23T01:19:16.762+0000 7fd246b12700 -1 osd.3 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.249541+0000 front 2023-06-23T01:18:50.249572+0000 (oldest deadline 2023-06-23T01:19:14.349769+0000) 2023-06-23T01:19:17.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:17 smithi134 conmon[143713]: 2023-06-23T01:19:17.091+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:17.876 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:17 smithi157 conmon[138256]: 2023-06-23T01:19:17.581+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:17.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:17 smithi157 conmon[145357]: 2023-06-23T01:19:17.516+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:18.260 INFO:journalctl@ceph.osd.3.smithi120.stdout:Jun 23 01:19:17 smithi120 conmon[103072]: 2023-06-23T01:19:17.807+0000 7fd246b12700 -1 osd.3 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.249541+0000 front 2023-06-23T01:18:50.249572+0000 (oldest deadline 2023-06-23T01:19:14.349769+0000) 2023-06-23T01:19:18.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:18 smithi134 conmon[143713]: 2023-06-23T01:19:18.066+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:18.523 INFO:journalctl@ceph.osd.5.smithi157.stdout:Jun 23 01:19:18 smithi157 conmon[131277]: 2023-06-23T01:19:18.188+0000 7fe05271c700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:53.598132+0000 front 2023-06-23T01:18:53.598194+0000 (oldest deadline 2023-06-23T01:19:17.698146+0000) 2023-06-23T01:19:18.876 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:18 smithi157 conmon[138256]: 2023-06-23T01:19:18.603+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:18.876 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:18 smithi157 conmon[145357]: 2023-06-23T01:19:18.519+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:18.953 INFO:journalctl@ceph.osd.3.smithi120.stdout:Jun 23 01:19:18 smithi120 bash[128666]: ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c-osd-3 2023-06-23T01:19:19.260 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:19 smithi120 conmon[110061]: 2023-06-23T01:19:19.079+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.689546+0000 front 2023-06-23T01:18:52.689652+0000 (oldest deadline 2023-06-23T01:19:18.589668+0000) 2023-06-23T01:19:19.260 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:18 smithi120 conmon[117204]: 2023-06-23T01:19:18.949+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:19.443 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:19 smithi134 conmon[143713]: 2023-06-23T01:19:19.038+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:19.517 INFO:journalctl@ceph.osd.5.smithi157.stdout:Jun 23 01:19:19 smithi157 conmon[131277]: 2023-06-23T01:19:19.144+0000 7fe05271c700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:53.598132+0000 front 2023-06-23T01:18:53.598194+0000 (oldest deadline 2023-06-23T01:19:17.698146+0000) 2023-06-23T01:19:19.876 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:19 smithi157 conmon[138256]: 2023-06-23T01:19:19.583+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:19.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:19 smithi157 conmon[145357]: 2023-06-23T01:19:19.513+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:20.010 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:19 smithi120 conmon[117204]: 2023-06-23T01:19:19.909+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:20.381 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:20 smithi120 conmon[110061]: 2023-06-23T01:19:20.107+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.689546+0000 front 2023-06-23T01:18:52.689652+0000 (oldest deadline 2023-06-23T01:19:18.589668+0000) 2023-06-23T01:19:20.443 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:19 smithi134 conmon[143713]: 2023-06-23T01:19:19.993+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:20.550 INFO:journalctl@ceph.osd.5.smithi157.stdout:Jun 23 01:19:20 smithi157 conmon[131277]: 2023-06-23T01:19:20.143+0000 7fe05271c700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:53.598132+0000 front 2023-06-23T01:18:53.598194+0000 (oldest deadline 2023-06-23T01:19:17.698146+0000) 2023-06-23T01:19:20.631 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@osd.3.service' 2023-06-23T01:19:20.686 INFO:journalctl@ceph.osd.3.smithi120.stdout:Jun 23 01:19:20 smithi120 systemd[1]: ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@osd.3.service: Succeeded. 2023-06-23T01:19:20.686 INFO:journalctl@ceph.osd.3.smithi120.stdout:Jun 23 01:19:20 smithi120 systemd[1]: Stopped Ceph osd.3 for 9efbfa8e-1162-11ee-9b2b-001a4aab830c. 2023-06-23T01:19:20.712 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T01:19:20.713 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-06-23T01:19:20.714 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-06-23T01:19:20.714 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@osd.4 2023-06-23T01:19:20.876 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:20 smithi157 conmon[138256]: 2023-06-23T01:19:20.597+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:20.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:20 smithi157 conmon[145357]: 2023-06-23T01:19:20.547+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:21.010 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:20 smithi120 conmon[117204]: 2023-06-23T01:19:20.872+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:21.070 INFO:journalctl@ceph.osd.4.smithi134.stdout:Jun 23 01:19:20 smithi134 systemd[1]: Stopping Ceph osd.4 for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:19:21.070 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:20 smithi134 conmon[143713]: 2023-06-23T01:19:20.983+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:21.376 INFO:journalctl@ceph.osd.5.smithi157.stdout:Jun 23 01:19:21 smithi157 conmon[131277]: 2023-06-23T01:19:21.099+0000 7fe05271c700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:53.598132+0000 front 2023-06-23T01:18:53.598194+0000 (oldest deadline 2023-06-23T01:19:17.698146+0000) 2023-06-23T01:19:21.443 INFO:journalctl@ceph.osd.4.smithi134.stdout:Jun 23 01:19:21 smithi134 conmon[129635]: 2023-06-23T01:19:21.066+0000 7fb424351700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T01:19:21.444 INFO:journalctl@ceph.osd.4.smithi134.stdout:Jun 23 01:19:21 smithi134 conmon[129635]: 2023-06-23T01:19:21.066+0000 7fb424351700 -1 osd.4 85 *** Got signal Terminated *** 2023-06-23T01:19:21.444 INFO:journalctl@ceph.osd.4.smithi134.stdout:Jun 23 01:19:21 smithi134 conmon[129635]: 2023-06-23T01:19:21.066+0000 7fb424351700 -1 osd.4 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-23T01:19:21.510 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:21 smithi120 conmon[110061]: 2023-06-23T01:19:21.132+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.689546+0000 front 2023-06-23T01:18:52.689652+0000 (oldest deadline 2023-06-23T01:19:18.589668+0000) 2023-06-23T01:19:21.876 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:21 smithi157 conmon[138256]: 2023-06-23T01:19:21.566+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:21.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:21 smithi157 conmon[145357]: 2023-06-23T01:19:21.554+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:22.166 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:21 smithi120 conmon[117204]: 2023-06-23T01:19:21.826+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:22.443 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:21 smithi134 conmon[143713]: 2023-06-23T01:19:21.984+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:22.510 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:22 smithi120 conmon[110061]: 2023-06-23T01:19:22.163+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.689546+0000 front 2023-06-23T01:18:52.689652+0000 (oldest deadline 2023-06-23T01:19:18.589668+0000) 2023-06-23T01:19:22.595 INFO:journalctl@ceph.osd.5.smithi157.stdout:Jun 23 01:19:22 smithi157 conmon[131277]: 2023-06-23T01:19:22.140+0000 7fe05271c700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:53.598132+0000 front 2023-06-23T01:18:53.598194+0000 (oldest deadline 2023-06-23T01:19:17.698146+0000) 2023-06-23T01:19:22.876 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:22 smithi157 conmon[138256]: 2023-06-23T01:19:22.614+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:22.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:22 smithi157 conmon[145357]: 2023-06-23T01:19:22.591+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:22.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:22 smithi157 conmon[145357]: 2023-06-23T01:19:22.591+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:19:23.195 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:22 smithi120 conmon[117204]: 2023-06-23T01:19:22.825+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:23.443 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:23 smithi134 conmon[143713]: 2023-06-23T01:19:23.022+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:23.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:23 smithi134 conmon[143713]: 2023-06-23T01:19:23.022+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:19:23.510 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:23 smithi120 conmon[110061]: 2023-06-23T01:19:23.192+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.689546+0000 front 2023-06-23T01:18:52.689652+0000 (oldest deadline 2023-06-23T01:19:18.589668+0000) 2023-06-23T01:19:23.561 INFO:journalctl@ceph.osd.5.smithi157.stdout:Jun 23 01:19:23 smithi157 conmon[131277]: 2023-06-23T01:19:23.124+0000 7fe05271c700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:53.598132+0000 front 2023-06-23T01:18:53.598194+0000 (oldest deadline 2023-06-23T01:19:17.698146+0000) 2023-06-23T01:19:23.876 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:23 smithi157 conmon[138256]: 2023-06-23T01:19:23.618+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:23.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:23 smithi157 conmon[145357]: 2023-06-23T01:19:23.558+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:23.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:23 smithi157 conmon[145357]: 2023-06-23T01:19:23.558+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:19:24.228 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:23 smithi120 conmon[117204]: 2023-06-23T01:19:23.776+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:24.443 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:24 smithi134 conmon[143713]: 2023-06-23T01:19:24.042+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:24.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:24 smithi134 conmon[143713]: 2023-06-23T01:19:24.042+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:19:24.510 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:24 smithi120 conmon[110061]: 2023-06-23T01:19:24.225+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.689546+0000 front 2023-06-23T01:18:52.689652+0000 (oldest deadline 2023-06-23T01:19:18.589668+0000) 2023-06-23T01:19:24.510 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:24 smithi120 conmon[110061]: 2023-06-23T01:19:24.225+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.590381+0000 front 2023-06-23T01:18:58.590229+0000 (oldest deadline 2023-06-23T01:19:23.290262+0000) 2023-06-23T01:19:24.568 INFO:journalctl@ceph.osd.5.smithi157.stdout:Jun 23 01:19:24 smithi157 conmon[131277]: 2023-06-23T01:19:24.134+0000 7fe05271c700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:53.598132+0000 front 2023-06-23T01:18:53.598194+0000 (oldest deadline 2023-06-23T01:19:17.698146+0000) 2023-06-23T01:19:24.569 INFO:journalctl@ceph.osd.5.smithi157.stdout:Jun 23 01:19:24 smithi157 conmon[131277]: 2023-06-23T01:19:24.134+0000 7fe05271c700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:57.698603+0000 front 2023-06-23T01:18:57.698592+0000 (oldest deadline 2023-06-23T01:19:23.598598+0000) 2023-06-23T01:19:24.876 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:24 smithi157 conmon[138256]: 2023-06-23T01:19:24.652+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:24.876 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:24 smithi157 conmon[145357]: 2023-06-23T01:19:24.565+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:24.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:24 smithi157 conmon[145357]: 2023-06-23T01:19:24.565+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:19:25.220 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:24 smithi120 conmon[117204]: 2023-06-23T01:19:24.804+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:25.443 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:25 smithi134 conmon[143713]: 2023-06-23T01:19:25.040+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:25.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:25 smithi134 conmon[143713]: 2023-06-23T01:19:25.040+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:19:25.510 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:25 smithi120 conmon[110061]: 2023-06-23T01:19:25.217+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.689546+0000 front 2023-06-23T01:18:52.689652+0000 (oldest deadline 2023-06-23T01:19:18.589668+0000) 2023-06-23T01:19:25.510 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:25 smithi120 conmon[110061]: 2023-06-23T01:19:25.217+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.590381+0000 front 2023-06-23T01:18:58.590229+0000 (oldest deadline 2023-06-23T01:19:23.290262+0000) 2023-06-23T01:19:25.543 INFO:journalctl@ceph.osd.5.smithi157.stdout:Jun 23 01:19:25 smithi157 conmon[131277]: 2023-06-23T01:19:25.141+0000 7fe05271c700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:53.598132+0000 front 2023-06-23T01:18:53.598194+0000 (oldest deadline 2023-06-23T01:19:17.698146+0000) 2023-06-23T01:19:25.544 INFO:journalctl@ceph.osd.5.smithi157.stdout:Jun 23 01:19:25 smithi157 conmon[131277]: 2023-06-23T01:19:25.141+0000 7fe05271c700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:57.698603+0000 front 2023-06-23T01:18:57.698592+0000 (oldest deadline 2023-06-23T01:19:23.598598+0000) 2023-06-23T01:19:25.876 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:25 smithi157 conmon[138256]: 2023-06-23T01:19:25.665+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:25.877 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:25 smithi157 conmon[138256]: 2023-06-23T01:19:25.665+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.766636+0000 front 2023-06-23T01:19:01.766694+0000 (oldest deadline 2023-06-23T01:19:24.666623+0000) 2023-06-23T01:19:25.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:25 smithi157 conmon[145357]: 2023-06-23T01:19:25.539+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:25.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:25 smithi157 conmon[145357]: 2023-06-23T01:19:25.539+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:19:26.222 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:25 smithi120 conmon[117204]: 2023-06-23T01:19:25.806+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:26.223 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:25 smithi120 conmon[117204]: 2023-06-23T01:19:25.806+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:19:26.444 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:25 smithi134 conmon[136657]: 2023-06-23T01:19:25.957+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.513557+0000 front 2023-06-23T01:19:01.513497+0000 (oldest deadline 2023-06-23T01:19:25.013698+0000) 2023-06-23T01:19:26.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:26 smithi134 conmon[143713]: 2023-06-23T01:19:26.039+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:26.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:26 smithi134 conmon[143713]: 2023-06-23T01:19:26.039+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:19:26.510 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:26 smithi120 conmon[110061]: 2023-06-23T01:19:26.220+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.689546+0000 front 2023-06-23T01:18:52.689652+0000 (oldest deadline 2023-06-23T01:19:18.589668+0000) 2023-06-23T01:19:26.510 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:26 smithi120 conmon[110061]: 2023-06-23T01:19:26.220+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.590381+0000 front 2023-06-23T01:18:58.590229+0000 (oldest deadline 2023-06-23T01:19:23.290262+0000) 2023-06-23T01:19:26.562 INFO:journalctl@ceph.osd.5.smithi157.stdout:Jun 23 01:19:26 smithi157 conmon[131277]: 2023-06-23T01:19:26.174+0000 7fe05271c700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:53.598132+0000 front 2023-06-23T01:18:53.598194+0000 (oldest deadline 2023-06-23T01:19:17.698146+0000) 2023-06-23T01:19:26.562 INFO:journalctl@ceph.osd.5.smithi157.stdout:Jun 23 01:19:26 smithi157 conmon[131277]: 2023-06-23T01:19:26.174+0000 7fe05271c700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:57.698603+0000 front 2023-06-23T01:18:57.698592+0000 (oldest deadline 2023-06-23T01:19:23.598598+0000) 2023-06-23T01:19:26.876 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:26 smithi157 conmon[138256]: 2023-06-23T01:19:26.701+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:26.877 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:26 smithi157 conmon[138256]: 2023-06-23T01:19:26.701+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.766636+0000 front 2023-06-23T01:19:01.766694+0000 (oldest deadline 2023-06-23T01:19:24.666623+0000) 2023-06-23T01:19:26.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:26 smithi157 conmon[145357]: 2023-06-23T01:19:26.558+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:26.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:26 smithi157 conmon[145357]: 2023-06-23T01:19:26.558+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:19:26.943 INFO:journalctl@ceph.osd.4.smithi134.stdout:Jun 23 01:19:26 smithi134 bash[152366]: ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c-osd-4 2023-06-23T01:19:27.260 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:26 smithi120 conmon[117204]: 2023-06-23T01:19:26.825+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:27.260 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:26 smithi120 conmon[117204]: 2023-06-23T01:19:26.825+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:19:27.443 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:26 smithi134 conmon[136657]: 2023-06-23T01:19:26.971+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.513557+0000 front 2023-06-23T01:19:01.513497+0000 (oldest deadline 2023-06-23T01:19:25.013698+0000) 2023-06-23T01:19:27.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:27 smithi134 conmon[143713]: 2023-06-23T01:19:27.034+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:27.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:27 smithi134 conmon[143713]: 2023-06-23T01:19:27.034+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:19:27.597 INFO:journalctl@ceph.osd.5.smithi157.stdout:Jun 23 01:19:27 smithi157 conmon[131277]: 2023-06-23T01:19:27.151+0000 7fe05271c700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:53.598132+0000 front 2023-06-23T01:18:53.598194+0000 (oldest deadline 2023-06-23T01:19:17.698146+0000) 2023-06-23T01:19:27.597 INFO:journalctl@ceph.osd.5.smithi157.stdout:Jun 23 01:19:27 smithi157 conmon[131277]: 2023-06-23T01:19:27.151+0000 7fe05271c700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:57.698603+0000 front 2023-06-23T01:18:57.698592+0000 (oldest deadline 2023-06-23T01:19:23.598598+0000) 2023-06-23T01:19:27.760 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:27 smithi120 conmon[110061]: 2023-06-23T01:19:27.263+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.689546+0000 front 2023-06-23T01:18:52.689652+0000 (oldest deadline 2023-06-23T01:19:18.589668+0000) 2023-06-23T01:19:27.760 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:27 smithi120 conmon[110061]: 2023-06-23T01:19:27.263+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.590381+0000 front 2023-06-23T01:18:58.590229+0000 (oldest deadline 2023-06-23T01:19:23.290262+0000) 2023-06-23T01:19:27.876 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:27 smithi157 conmon[138256]: 2023-06-23T01:19:27.725+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:27.876 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:27 smithi157 conmon[138256]: 2023-06-23T01:19:27.725+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.766636+0000 front 2023-06-23T01:19:01.766694+0000 (oldest deadline 2023-06-23T01:19:24.666623+0000) 2023-06-23T01:19:27.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:27 smithi157 conmon[145357]: 2023-06-23T01:19:27.594+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:27.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:27 smithi157 conmon[145357]: 2023-06-23T01:19:27.594+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:19:28.260 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:27 smithi120 conmon[117204]: 2023-06-23T01:19:27.798+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:28.260 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:27 smithi120 conmon[117204]: 2023-06-23T01:19:27.798+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:19:28.444 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:27 smithi134 conmon[136657]: 2023-06-23T01:19:27.971+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.513557+0000 front 2023-06-23T01:19:01.513497+0000 (oldest deadline 2023-06-23T01:19:25.013698+0000) 2023-06-23T01:19:28.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:27 smithi134 conmon[143713]: 2023-06-23T01:19:27.984+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:28.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:27 smithi134 conmon[143713]: 2023-06-23T01:19:27.984+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:19:28.567 INFO:journalctl@ceph.osd.5.smithi157.stdout:Jun 23 01:19:28 smithi157 conmon[131277]: 2023-06-23T01:19:28.190+0000 7fe05271c700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:53.598132+0000 front 2023-06-23T01:18:53.598194+0000 (oldest deadline 2023-06-23T01:19:17.698146+0000) 2023-06-23T01:19:28.567 INFO:journalctl@ceph.osd.5.smithi157.stdout:Jun 23 01:19:28 smithi157 conmon[131277]: 2023-06-23T01:19:28.190+0000 7fe05271c700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:57.698603+0000 front 2023-06-23T01:18:57.698592+0000 (oldest deadline 2023-06-23T01:19:23.598598+0000) 2023-06-23T01:19:28.606 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@osd.4.service' 2023-06-23T01:19:28.643 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T01:19:28.644 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-06-23T01:19:28.645 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-06-23T01:19:28.645 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@osd.5 2023-06-23T01:19:28.760 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:28 smithi120 conmon[110061]: 2023-06-23T01:19:28.257+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.689546+0000 front 2023-06-23T01:18:52.689652+0000 (oldest deadline 2023-06-23T01:19:18.589668+0000) 2023-06-23T01:19:28.761 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:28 smithi120 conmon[110061]: 2023-06-23T01:19:28.257+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.590381+0000 front 2023-06-23T01:18:58.590229+0000 (oldest deadline 2023-06-23T01:19:23.290262+0000) 2023-06-23T01:19:28.876 INFO:journalctl@ceph.osd.5.smithi157.stdout:Jun 23 01:19:28 smithi157 systemd[1]: Stopping Ceph osd.5 for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:19:28.877 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:28 smithi157 conmon[138256]: 2023-06-23T01:19:28.752+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:28.878 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:28 smithi157 conmon[138256]: 2023-06-23T01:19:28.752+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.766636+0000 front 2023-06-23T01:19:01.766694+0000 (oldest deadline 2023-06-23T01:19:24.666623+0000) 2023-06-23T01:19:28.878 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:28 smithi157 conmon[145357]: 2023-06-23T01:19:28.564+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:28.879 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:28 smithi157 conmon[145357]: 2023-06-23T01:19:28.564+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:19:29.193 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:28 smithi134 conmon[136657]: 2023-06-23T01:19:28.972+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.513557+0000 front 2023-06-23T01:19:01.513497+0000 (oldest deadline 2023-06-23T01:19:25.013698+0000) 2023-06-23T01:19:29.194 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:28 smithi134 conmon[143713]: 2023-06-23T01:19:28.938+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:29.194 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:28 smithi134 conmon[143713]: 2023-06-23T01:19:28.938+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:19:29.247 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:28 smithi120 conmon[117204]: 2023-06-23T01:19:28.794+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:29.247 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:28 smithi120 conmon[117204]: 2023-06-23T01:19:28.794+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:19:29.376 INFO:journalctl@ceph.osd.5.smithi157.stdout:Jun 23 01:19:28 smithi157 conmon[131277]: 2023-06-23T01:19:28.995+0000 7fe057120700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T01:19:29.377 INFO:journalctl@ceph.osd.5.smithi157.stdout:Jun 23 01:19:28 smithi157 conmon[131277]: 2023-06-23T01:19:28.995+0000 7fe057120700 -1 osd.5 85 *** Got signal Terminated *** 2023-06-23T01:19:29.377 INFO:journalctl@ceph.osd.5.smithi157.stdout:Jun 23 01:19:28 smithi157 conmon[131277]: 2023-06-23T01:19:28.995+0000 7fe057120700 -1 osd.5 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-23T01:19:29.377 INFO:journalctl@ceph.osd.5.smithi157.stdout:Jun 23 01:19:29 smithi157 conmon[131277]: 2023-06-23T01:19:29.208+0000 7fe05271c700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:53.598132+0000 front 2023-06-23T01:18:53.598194+0000 (oldest deadline 2023-06-23T01:19:17.698146+0000) 2023-06-23T01:19:29.377 INFO:journalctl@ceph.osd.5.smithi157.stdout:Jun 23 01:19:29 smithi157 conmon[131277]: 2023-06-23T01:19:29.208+0000 7fe05271c700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:57.698603+0000 front 2023-06-23T01:18:57.698592+0000 (oldest deadline 2023-06-23T01:19:23.598598+0000) 2023-06-23T01:19:29.510 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:29 smithi120 conmon[110061]: 2023-06-23T01:19:29.244+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.689546+0000 front 2023-06-23T01:18:52.689652+0000 (oldest deadline 2023-06-23T01:19:18.589668+0000) 2023-06-23T01:19:29.510 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:29 smithi120 conmon[110061]: 2023-06-23T01:19:29.244+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.590381+0000 front 2023-06-23T01:18:58.590229+0000 (oldest deadline 2023-06-23T01:19:23.290262+0000) 2023-06-23T01:19:29.876 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:29 smithi157 conmon[145357]: 2023-06-23T01:19:29.556+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:29.876 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:29 smithi157 conmon[145357]: 2023-06-23T01:19:29.556+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:19:29.877 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:29 smithi157 conmon[138256]: 2023-06-23T01:19:29.797+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:29.877 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:29 smithi157 conmon[138256]: 2023-06-23T01:19:29.797+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.766636+0000 front 2023-06-23T01:19:01.766694+0000 (oldest deadline 2023-06-23T01:19:24.666623+0000) 2023-06-23T01:19:30.193 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:29 smithi134 conmon[136657]: 2023-06-23T01:19:29.998+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.513557+0000 front 2023-06-23T01:19:01.513497+0000 (oldest deadline 2023-06-23T01:19:25.013698+0000) 2023-06-23T01:19:30.194 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:29 smithi134 conmon[143713]: 2023-06-23T01:19:29.902+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:30.194 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:29 smithi134 conmon[143713]: 2023-06-23T01:19:29.902+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:19:30.224 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:29 smithi120 conmon[117204]: 2023-06-23T01:19:29.790+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:30.225 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:29 smithi120 conmon[117204]: 2023-06-23T01:19:29.790+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:19:30.510 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:30 smithi120 conmon[110061]: 2023-06-23T01:19:30.222+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.689546+0000 front 2023-06-23T01:18:52.689652+0000 (oldest deadline 2023-06-23T01:19:18.589668+0000) 2023-06-23T01:19:30.510 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:30 smithi120 conmon[110061]: 2023-06-23T01:19:30.222+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.590381+0000 front 2023-06-23T01:18:58.590229+0000 (oldest deadline 2023-06-23T01:19:23.290262+0000) 2023-06-23T01:19:30.572 INFO:journalctl@ceph.osd.5.smithi157.stdout:Jun 23 01:19:30 smithi157 conmon[131277]: 2023-06-23T01:19:30.243+0000 7fe05271c700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:53.598132+0000 front 2023-06-23T01:18:53.598194+0000 (oldest deadline 2023-06-23T01:19:17.698146+0000) 2023-06-23T01:19:30.572 INFO:journalctl@ceph.osd.5.smithi157.stdout:Jun 23 01:19:30 smithi157 conmon[131277]: 2023-06-23T01:19:30.243+0000 7fe05271c700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:57.698603+0000 front 2023-06-23T01:18:57.698592+0000 (oldest deadline 2023-06-23T01:19:23.598598+0000) 2023-06-23T01:19:30.845 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:30 smithi157 conmon[145357]: 2023-06-23T01:19:30.569+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:30.845 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:30 smithi157 conmon[145357]: 2023-06-23T01:19:30.569+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:19:31.126 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:30 smithi157 conmon[138256]: 2023-06-23T01:19:30.842+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:31.126 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:30 smithi157 conmon[138256]: 2023-06-23T01:19:30.842+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.766636+0000 front 2023-06-23T01:19:01.766694+0000 (oldest deadline 2023-06-23T01:19:24.666623+0000) 2023-06-23T01:19:31.126 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:30 smithi157 conmon[138256]: 2023-06-23T01:19:30.842+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.167466+0000 front 2023-06-23T01:19:05.167527+0000 (oldest deadline 2023-06-23T01:19:30.467672+0000) 2023-06-23T01:19:31.194 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:30 smithi134 conmon[136657]: 2023-06-23T01:19:30.961+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.513557+0000 front 2023-06-23T01:19:01.513497+0000 (oldest deadline 2023-06-23T01:19:25.013698+0000) 2023-06-23T01:19:31.194 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:30 smithi134 conmon[143713]: 2023-06-23T01:19:30.897+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:31.195 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:30 smithi134 conmon[143713]: 2023-06-23T01:19:30.897+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:19:31.218 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:30 smithi120 conmon[117204]: 2023-06-23T01:19:30.829+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:31.219 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:30 smithi120 conmon[117204]: 2023-06-23T01:19:30.829+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:19:31.510 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:31 smithi120 conmon[110061]: 2023-06-23T01:19:31.214+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.689546+0000 front 2023-06-23T01:18:52.689652+0000 (oldest deadline 2023-06-23T01:19:18.589668+0000) 2023-06-23T01:19:31.511 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:31 smithi120 conmon[110061]: 2023-06-23T01:19:31.214+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.590381+0000 front 2023-06-23T01:18:58.590229+0000 (oldest deadline 2023-06-23T01:19:23.290262+0000) 2023-06-23T01:19:31.563 INFO:journalctl@ceph.osd.5.smithi157.stdout:Jun 23 01:19:31 smithi157 conmon[131277]: 2023-06-23T01:19:31.228+0000 7fe05271c700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:53.598132+0000 front 2023-06-23T01:18:53.598194+0000 (oldest deadline 2023-06-23T01:19:17.698146+0000) 2023-06-23T01:19:31.563 INFO:journalctl@ceph.osd.5.smithi157.stdout:Jun 23 01:19:31 smithi157 conmon[131277]: 2023-06-23T01:19:31.228+0000 7fe05271c700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:57.698603+0000 front 2023-06-23T01:18:57.698592+0000 (oldest deadline 2023-06-23T01:19:23.598598+0000) 2023-06-23T01:19:31.874 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:31 smithi157 conmon[145357]: 2023-06-23T01:19:31.560+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:31.874 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:31 smithi157 conmon[145357]: 2023-06-23T01:19:31.560+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:19:32.126 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:31 smithi157 conmon[138256]: 2023-06-23T01:19:31.870+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:32.126 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:31 smithi157 conmon[138256]: 2023-06-23T01:19:31.870+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.766636+0000 front 2023-06-23T01:19:01.766694+0000 (oldest deadline 2023-06-23T01:19:24.666623+0000) 2023-06-23T01:19:32.127 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:31 smithi157 conmon[138256]: 2023-06-23T01:19:31.870+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.167466+0000 front 2023-06-23T01:19:05.167527+0000 (oldest deadline 2023-06-23T01:19:30.467672+0000) 2023-06-23T01:19:32.194 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:31 smithi134 conmon[136657]: 2023-06-23T01:19:31.918+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.513557+0000 front 2023-06-23T01:19:01.513497+0000 (oldest deadline 2023-06-23T01:19:25.013698+0000) 2023-06-23T01:19:32.194 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:31 smithi134 conmon[143713]: 2023-06-23T01:19:31.932+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:32.194 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:31 smithi134 conmon[143713]: 2023-06-23T01:19:31.932+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:19:32.259 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:31 smithi120 conmon[117204]: 2023-06-23T01:19:31.823+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:32.260 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:31 smithi120 conmon[117204]: 2023-06-23T01:19:31.823+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:19:32.260 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:31 smithi120 conmon[117204]: 2023-06-23T01:19:31.823+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.597925+0000 front 2023-06-23T01:19:05.597907+0000 (oldest deadline 2023-06-23T01:19:31.497672+0000) 2023-06-23T01:19:32.510 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:32 smithi120 conmon[110061]: 2023-06-23T01:19:32.256+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.689546+0000 front 2023-06-23T01:18:52.689652+0000 (oldest deadline 2023-06-23T01:19:18.589668+0000) 2023-06-23T01:19:32.510 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:32 smithi120 conmon[110061]: 2023-06-23T01:19:32.256+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.590381+0000 front 2023-06-23T01:18:58.590229+0000 (oldest deadline 2023-06-23T01:19:23.290262+0000) 2023-06-23T01:19:32.581 INFO:journalctl@ceph.osd.5.smithi157.stdout:Jun 23 01:19:32 smithi157 conmon[131277]: 2023-06-23T01:19:32.273+0000 7fe05271c700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:53.598132+0000 front 2023-06-23T01:18:53.598194+0000 (oldest deadline 2023-06-23T01:19:17.698146+0000) 2023-06-23T01:19:32.581 INFO:journalctl@ceph.osd.5.smithi157.stdout:Jun 23 01:19:32 smithi157 conmon[131277]: 2023-06-23T01:19:32.273+0000 7fe05271c700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:57.698603+0000 front 2023-06-23T01:18:57.698592+0000 (oldest deadline 2023-06-23T01:19:23.598598+0000) 2023-06-23T01:19:32.876 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:32 smithi157 conmon[145357]: 2023-06-23T01:19:32.578+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:32.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:32 smithi157 conmon[145357]: 2023-06-23T01:19:32.578+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:19:33.194 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:32 smithi134 conmon[136657]: 2023-06-23T01:19:32.909+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.513557+0000 front 2023-06-23T01:19:01.513497+0000 (oldest deadline 2023-06-23T01:19:25.013698+0000) 2023-06-23T01:19:33.194 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:32 smithi134 conmon[143713]: 2023-06-23T01:19:32.982+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:33.194 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:32 smithi134 conmon[143713]: 2023-06-23T01:19:32.982+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:19:33.194 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:32 smithi134 conmon[143713]: 2023-06-23T01:19:32.982+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:19:33.246 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:32 smithi120 conmon[117204]: 2023-06-23T01:19:32.791+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:33.247 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:32 smithi120 conmon[117204]: 2023-06-23T01:19:32.791+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:19:33.247 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:32 smithi120 conmon[117204]: 2023-06-23T01:19:32.791+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.597925+0000 front 2023-06-23T01:19:05.597907+0000 (oldest deadline 2023-06-23T01:19:31.497672+0000) 2023-06-23T01:19:33.300 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:32 smithi157 conmon[138256]: 2023-06-23T01:19:32.907+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:33.300 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:32 smithi157 conmon[138256]: 2023-06-23T01:19:32.907+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.766636+0000 front 2023-06-23T01:19:01.766694+0000 (oldest deadline 2023-06-23T01:19:24.666623+0000) 2023-06-23T01:19:33.301 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:32 smithi157 conmon[138256]: 2023-06-23T01:19:32.907+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.167466+0000 front 2023-06-23T01:19:05.167527+0000 (oldest deadline 2023-06-23T01:19:30.467672+0000) 2023-06-23T01:19:33.510 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:33 smithi120 conmon[110061]: 2023-06-23T01:19:33.243+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.689546+0000 front 2023-06-23T01:18:52.689652+0000 (oldest deadline 2023-06-23T01:19:18.589668+0000) 2023-06-23T01:19:33.510 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:33 smithi120 conmon[110061]: 2023-06-23T01:19:33.243+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.590381+0000 front 2023-06-23T01:18:58.590229+0000 (oldest deadline 2023-06-23T01:19:23.290262+0000) 2023-06-23T01:19:33.511 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:33 smithi120 conmon[110061]: 2023-06-23T01:19:33.243+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:09.692142+0000 front 2023-06-23T01:19:09.692161+0000 (oldest deadline 2023-06-23T01:19:32.592217+0000) 2023-06-23T01:19:33.557 INFO:journalctl@ceph.osd.5.smithi157.stdout:Jun 23 01:19:33 smithi157 conmon[131277]: 2023-06-23T01:19:33.297+0000 7fe05271c700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:53.598132+0000 front 2023-06-23T01:18:53.598194+0000 (oldest deadline 2023-06-23T01:19:17.698146+0000) 2023-06-23T01:19:33.557 INFO:journalctl@ceph.osd.5.smithi157.stdout:Jun 23 01:19:33 smithi157 conmon[131277]: 2023-06-23T01:19:33.297+0000 7fe05271c700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:57.698603+0000 front 2023-06-23T01:18:57.698592+0000 (oldest deadline 2023-06-23T01:19:23.598598+0000) 2023-06-23T01:19:33.876 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:33 smithi157 conmon[145357]: 2023-06-23T01:19:33.554+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:33.876 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:33 smithi157 conmon[145357]: 2023-06-23T01:19:33.554+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:19:33.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:33 smithi157 conmon[145357]: 2023-06-23T01:19:33.554+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:19:34.193 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:33 smithi134 conmon[136657]: 2023-06-23T01:19:33.931+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.513557+0000 front 2023-06-23T01:19:01.513497+0000 (oldest deadline 2023-06-23T01:19:25.013698+0000) 2023-06-23T01:19:34.194 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:34 smithi134 conmon[143713]: 2023-06-23T01:19:34.004+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:34.194 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:34 smithi134 conmon[143713]: 2023-06-23T01:19:34.004+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:19:34.194 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:34 smithi134 conmon[143713]: 2023-06-23T01:19:34.004+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:19:34.225 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:33 smithi120 conmon[117204]: 2023-06-23T01:19:33.777+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:34.225 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:33 smithi120 conmon[117204]: 2023-06-23T01:19:33.777+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:19:34.225 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:33 smithi120 conmon[117204]: 2023-06-23T01:19:33.777+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.597925+0000 front 2023-06-23T01:19:05.597907+0000 (oldest deadline 2023-06-23T01:19:31.497672+0000) 2023-06-23T01:19:34.376 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:33 smithi157 conmon[138256]: 2023-06-23T01:19:33.893+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:34.376 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:33 smithi157 conmon[138256]: 2023-06-23T01:19:33.893+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.766636+0000 front 2023-06-23T01:19:01.766694+0000 (oldest deadline 2023-06-23T01:19:24.666623+0000) 2023-06-23T01:19:34.377 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:33 smithi157 conmon[138256]: 2023-06-23T01:19:33.893+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.167466+0000 front 2023-06-23T01:19:05.167527+0000 (oldest deadline 2023-06-23T01:19:30.467672+0000) 2023-06-23T01:19:34.510 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:34 smithi120 conmon[110061]: 2023-06-23T01:19:34.220+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.689546+0000 front 2023-06-23T01:18:52.689652+0000 (oldest deadline 2023-06-23T01:19:18.589668+0000) 2023-06-23T01:19:34.511 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:34 smithi120 conmon[110061]: 2023-06-23T01:19:34.220+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.590381+0000 front 2023-06-23T01:18:58.590229+0000 (oldest deadline 2023-06-23T01:19:23.290262+0000) 2023-06-23T01:19:34.511 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:34 smithi120 conmon[110061]: 2023-06-23T01:19:34.220+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:09.692142+0000 front 2023-06-23T01:19:09.692161+0000 (oldest deadline 2023-06-23T01:19:32.592217+0000) 2023-06-23T01:19:34.842 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:34 smithi157 conmon[145357]: 2023-06-23T01:19:34.576+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:34.842 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:34 smithi157 conmon[145357]: 2023-06-23T01:19:34.576+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:19:34.843 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:34 smithi157 conmon[145357]: 2023-06-23T01:19:34.576+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:19:35.126 INFO:journalctl@ceph.osd.5.smithi157.stdout:Jun 23 01:19:34 smithi157 bash[154449]: ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c-osd-5 2023-06-23T01:19:35.127 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:34 smithi157 conmon[138256]: 2023-06-23T01:19:34.939+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:35.127 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:34 smithi157 conmon[138256]: 2023-06-23T01:19:34.939+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.766636+0000 front 2023-06-23T01:19:01.766694+0000 (oldest deadline 2023-06-23T01:19:24.666623+0000) 2023-06-23T01:19:35.127 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:34 smithi157 conmon[138256]: 2023-06-23T01:19:34.939+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.167466+0000 front 2023-06-23T01:19:05.167527+0000 (oldest deadline 2023-06-23T01:19:30.467672+0000) 2023-06-23T01:19:35.242 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:34 smithi120 conmon[117204]: 2023-06-23T01:19:34.789+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:35.242 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:34 smithi120 conmon[117204]: 2023-06-23T01:19:34.789+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:19:35.243 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:34 smithi120 conmon[117204]: 2023-06-23T01:19:34.789+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.597925+0000 front 2023-06-23T01:19:05.597907+0000 (oldest deadline 2023-06-23T01:19:31.497672+0000) 2023-06-23T01:19:35.444 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:34 smithi134 conmon[136657]: 2023-06-23T01:19:34.961+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.513557+0000 front 2023-06-23T01:19:01.513497+0000 (oldest deadline 2023-06-23T01:19:25.013698+0000) 2023-06-23T01:19:35.444 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:34 smithi134 conmon[136657]: 2023-06-23T01:19:34.961+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:09.615111+0000 front 2023-06-23T01:19:09.615136+0000 (oldest deadline 2023-06-23T01:19:34.915349+0000) 2023-06-23T01:19:35.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:35 smithi134 conmon[143713]: 2023-06-23T01:19:34.999+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:35.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:35 smithi134 conmon[143713]: 2023-06-23T01:19:34.999+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:19:35.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:35 smithi134 conmon[143713]: 2023-06-23T01:19:34.999+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:19:35.510 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:35 smithi120 conmon[110061]: 2023-06-23T01:19:35.238+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.689546+0000 front 2023-06-23T01:18:52.689652+0000 (oldest deadline 2023-06-23T01:19:18.589668+0000) 2023-06-23T01:19:35.511 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:35 smithi120 conmon[110061]: 2023-06-23T01:19:35.238+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.590381+0000 front 2023-06-23T01:18:58.590229+0000 (oldest deadline 2023-06-23T01:19:23.290262+0000) 2023-06-23T01:19:35.511 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:35 smithi120 conmon[110061]: 2023-06-23T01:19:35.238+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:09.692142+0000 front 2023-06-23T01:19:09.692161+0000 (oldest deadline 2023-06-23T01:19:32.592217+0000) 2023-06-23T01:19:35.866 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:35 smithi157 conmon[145357]: 2023-06-23T01:19:35.526+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:35.867 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:35 smithi157 conmon[145357]: 2023-06-23T01:19:35.526+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:19:35.867 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:35 smithi157 conmon[145357]: 2023-06-23T01:19:35.526+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:19:36.126 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:35 smithi157 conmon[138256]: 2023-06-23T01:19:35.955+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:36.126 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:35 smithi157 conmon[138256]: 2023-06-23T01:19:35.955+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.766636+0000 front 2023-06-23T01:19:01.766694+0000 (oldest deadline 2023-06-23T01:19:24.666623+0000) 2023-06-23T01:19:36.127 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:35 smithi157 conmon[138256]: 2023-06-23T01:19:35.955+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.167466+0000 front 2023-06-23T01:19:05.167527+0000 (oldest deadline 2023-06-23T01:19:30.467672+0000) 2023-06-23T01:19:36.220 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:35 smithi120 conmon[117204]: 2023-06-23T01:19:35.774+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:36.221 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:35 smithi120 conmon[117204]: 2023-06-23T01:19:35.774+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:19:36.221 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:35 smithi120 conmon[117204]: 2023-06-23T01:19:35.774+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.597925+0000 front 2023-06-23T01:19:05.597907+0000 (oldest deadline 2023-06-23T01:19:31.497672+0000) 2023-06-23T01:19:36.444 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:35 smithi134 conmon[136657]: 2023-06-23T01:19:35.964+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.513557+0000 front 2023-06-23T01:19:01.513497+0000 (oldest deadline 2023-06-23T01:19:25.013698+0000) 2023-06-23T01:19:36.444 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:35 smithi134 conmon[136657]: 2023-06-23T01:19:35.964+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:09.615111+0000 front 2023-06-23T01:19:09.615136+0000 (oldest deadline 2023-06-23T01:19:34.915349+0000) 2023-06-23T01:19:36.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:35 smithi134 conmon[143713]: 2023-06-23T01:19:35.958+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:36.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:35 smithi134 conmon[143713]: 2023-06-23T01:19:35.958+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:19:36.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:35 smithi134 conmon[143713]: 2023-06-23T01:19:35.958+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:19:36.510 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:36 smithi120 conmon[110061]: 2023-06-23T01:19:36.217+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.689546+0000 front 2023-06-23T01:18:52.689652+0000 (oldest deadline 2023-06-23T01:19:18.589668+0000) 2023-06-23T01:19:36.510 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:36 smithi120 conmon[110061]: 2023-06-23T01:19:36.217+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.590381+0000 front 2023-06-23T01:18:58.590229+0000 (oldest deadline 2023-06-23T01:19:23.290262+0000) 2023-06-23T01:19:36.511 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:36 smithi120 conmon[110061]: 2023-06-23T01:19:36.217+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:09.692142+0000 front 2023-06-23T01:19:09.692161+0000 (oldest deadline 2023-06-23T01:19:32.592217+0000) 2023-06-23T01:19:36.606 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@osd.5.service' 2023-06-23T01:19:36.645 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T01:19:36.645 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-06-23T01:19:36.646 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-06-23T01:19:36.646 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@osd.6 2023-06-23T01:19:36.876 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:36 smithi157 conmon[145357]: 2023-06-23T01:19:36.565+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:36.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:36 smithi157 conmon[145357]: 2023-06-23T01:19:36.565+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:19:36.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:36 smithi157 conmon[145357]: 2023-06-23T01:19:36.565+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:19:37.011 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:36 smithi120 systemd[1]: Stopping Ceph osd.6 for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:19:37.011 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:36 smithi120 conmon[117204]: 2023-06-23T01:19:36.736+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:37.011 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:36 smithi120 conmon[117204]: 2023-06-23T01:19:36.736+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:19:37.011 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:36 smithi120 conmon[117204]: 2023-06-23T01:19:36.736+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.597925+0000 front 2023-06-23T01:19:05.597907+0000 (oldest deadline 2023-06-23T01:19:31.497672+0000) 2023-06-23T01:19:37.376 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:36 smithi157 conmon[138256]: 2023-06-23T01:19:36.919+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:37.377 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:36 smithi157 conmon[138256]: 2023-06-23T01:19:36.919+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.766636+0000 front 2023-06-23T01:19:01.766694+0000 (oldest deadline 2023-06-23T01:19:24.666623+0000) 2023-06-23T01:19:37.377 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:36 smithi157 conmon[138256]: 2023-06-23T01:19:36.919+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.167466+0000 front 2023-06-23T01:19:05.167527+0000 (oldest deadline 2023-06-23T01:19:30.467672+0000) 2023-06-23T01:19:37.448 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:36 smithi134 conmon[136657]: 2023-06-23T01:19:36.969+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.513557+0000 front 2023-06-23T01:19:01.513497+0000 (oldest deadline 2023-06-23T01:19:25.013698+0000) 2023-06-23T01:19:37.449 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:36 smithi134 conmon[136657]: 2023-06-23T01:19:36.969+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:09.615111+0000 front 2023-06-23T01:19:09.615136+0000 (oldest deadline 2023-06-23T01:19:34.915349+0000) 2023-06-23T01:19:37.449 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:37 smithi134 conmon[143713]: 2023-06-23T01:19:37.008+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:37.449 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:37 smithi134 conmon[143713]: 2023-06-23T01:19:37.008+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:19:37.450 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:37 smithi134 conmon[143713]: 2023-06-23T01:19:37.008+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:19:37.510 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:37 smithi120 conmon[110061]: 2023-06-23T01:19:37.161+0000 7f8bd1b82700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T01:19:37.510 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:37 smithi120 conmon[110061]: 2023-06-23T01:19:37.161+0000 7f8bd1b82700 -1 osd.6 85 *** Got signal Terminated *** 2023-06-23T01:19:37.510 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:37 smithi120 conmon[110061]: 2023-06-23T01:19:37.161+0000 7f8bd1b82700 -1 osd.6 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-23T01:19:37.511 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:37 smithi120 conmon[110061]: 2023-06-23T01:19:37.209+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.689546+0000 front 2023-06-23T01:18:52.689652+0000 (oldest deadline 2023-06-23T01:19:18.589668+0000) 2023-06-23T01:19:37.511 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:37 smithi120 conmon[110061]: 2023-06-23T01:19:37.209+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.590381+0000 front 2023-06-23T01:18:58.590229+0000 (oldest deadline 2023-06-23T01:19:23.290262+0000) 2023-06-23T01:19:37.511 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:37 smithi120 conmon[110061]: 2023-06-23T01:19:37.209+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:09.692142+0000 front 2023-06-23T01:19:09.692161+0000 (oldest deadline 2023-06-23T01:19:32.592217+0000) 2023-06-23T01:19:37.876 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:37 smithi157 conmon[145357]: 2023-06-23T01:19:37.541+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:37.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:37 smithi157 conmon[145357]: 2023-06-23T01:19:37.541+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:19:37.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:37 smithi157 conmon[145357]: 2023-06-23T01:19:37.541+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:19:38.010 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:37 smithi120 conmon[117204]: 2023-06-23T01:19:37.697+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:38.010 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:37 smithi120 conmon[117204]: 2023-06-23T01:19:37.697+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:19:38.010 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:37 smithi120 conmon[117204]: 2023-06-23T01:19:37.697+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.597925+0000 front 2023-06-23T01:19:05.597907+0000 (oldest deadline 2023-06-23T01:19:31.497672+0000) 2023-06-23T01:19:38.376 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:37 smithi157 conmon[138256]: 2023-06-23T01:19:37.894+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:38.376 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:37 smithi157 conmon[138256]: 2023-06-23T01:19:37.894+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.766636+0000 front 2023-06-23T01:19:01.766694+0000 (oldest deadline 2023-06-23T01:19:24.666623+0000) 2023-06-23T01:19:38.376 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:37 smithi157 conmon[138256]: 2023-06-23T01:19:37.894+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.167466+0000 front 2023-06-23T01:19:05.167527+0000 (oldest deadline 2023-06-23T01:19:30.467672+0000) 2023-06-23T01:19:38.443 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:37 smithi134 conmon[136657]: 2023-06-23T01:19:37.942+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.513557+0000 front 2023-06-23T01:19:01.513497+0000 (oldest deadline 2023-06-23T01:19:25.013698+0000) 2023-06-23T01:19:38.444 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:37 smithi134 conmon[136657]: 2023-06-23T01:19:37.942+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:09.615111+0000 front 2023-06-23T01:19:09.615136+0000 (oldest deadline 2023-06-23T01:19:34.915349+0000) 2023-06-23T01:19:38.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:38 smithi134 conmon[143713]: 2023-06-23T01:19:38.044+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:38.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:38 smithi134 conmon[143713]: 2023-06-23T01:19:38.044+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:19:38.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:38 smithi134 conmon[143713]: 2023-06-23T01:19:38.044+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:19:38.510 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:38 smithi120 conmon[110061]: 2023-06-23T01:19:38.231+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.689546+0000 front 2023-06-23T01:18:52.689652+0000 (oldest deadline 2023-06-23T01:19:18.589668+0000) 2023-06-23T01:19:38.510 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:38 smithi120 conmon[110061]: 2023-06-23T01:19:38.231+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.590381+0000 front 2023-06-23T01:18:58.590229+0000 (oldest deadline 2023-06-23T01:19:23.290262+0000) 2023-06-23T01:19:38.511 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:38 smithi120 conmon[110061]: 2023-06-23T01:19:38.231+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:09.692142+0000 front 2023-06-23T01:19:09.692161+0000 (oldest deadline 2023-06-23T01:19:32.592217+0000) 2023-06-23T01:19:38.876 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:38 smithi157 conmon[145357]: 2023-06-23T01:19:38.552+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:38.876 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:38 smithi157 conmon[145357]: 2023-06-23T01:19:38.552+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:19:38.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:38 smithi157 conmon[145357]: 2023-06-23T01:19:38.552+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:19:39.010 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:38 smithi120 conmon[117204]: 2023-06-23T01:19:38.656+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:39.010 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:38 smithi120 conmon[117204]: 2023-06-23T01:19:38.656+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:19:39.011 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:38 smithi120 conmon[117204]: 2023-06-23T01:19:38.656+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.597925+0000 front 2023-06-23T01:19:05.597907+0000 (oldest deadline 2023-06-23T01:19:31.497672+0000) 2023-06-23T01:19:39.376 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:38 smithi157 conmon[138256]: 2023-06-23T01:19:38.930+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:39.376 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:38 smithi157 conmon[138256]: 2023-06-23T01:19:38.930+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.766636+0000 front 2023-06-23T01:19:01.766694+0000 (oldest deadline 2023-06-23T01:19:24.666623+0000) 2023-06-23T01:19:39.377 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:38 smithi157 conmon[138256]: 2023-06-23T01:19:38.930+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.167466+0000 front 2023-06-23T01:19:05.167527+0000 (oldest deadline 2023-06-23T01:19:30.467672+0000) 2023-06-23T01:19:39.443 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:38 smithi134 conmon[136657]: 2023-06-23T01:19:38.968+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.513557+0000 front 2023-06-23T01:19:01.513497+0000 (oldest deadline 2023-06-23T01:19:25.013698+0000) 2023-06-23T01:19:39.444 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:38 smithi134 conmon[136657]: 2023-06-23T01:19:38.968+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:09.615111+0000 front 2023-06-23T01:19:09.615136+0000 (oldest deadline 2023-06-23T01:19:34.915349+0000) 2023-06-23T01:19:39.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:38 smithi134 conmon[143713]: 2023-06-23T01:19:38.998+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:39.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:38 smithi134 conmon[143713]: 2023-06-23T01:19:38.998+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:19:39.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:38 smithi134 conmon[143713]: 2023-06-23T01:19:38.998+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:19:39.510 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:39 smithi120 conmon[110061]: 2023-06-23T01:19:39.209+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.689546+0000 front 2023-06-23T01:18:52.689652+0000 (oldest deadline 2023-06-23T01:19:18.589668+0000) 2023-06-23T01:19:39.510 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:39 smithi120 conmon[110061]: 2023-06-23T01:19:39.209+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.590381+0000 front 2023-06-23T01:18:58.590229+0000 (oldest deadline 2023-06-23T01:19:23.290262+0000) 2023-06-23T01:19:39.510 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:39 smithi120 conmon[110061]: 2023-06-23T01:19:39.209+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:09.692142+0000 front 2023-06-23T01:19:09.692161+0000 (oldest deadline 2023-06-23T01:19:32.592217+0000) 2023-06-23T01:19:39.876 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:39 smithi157 conmon[145357]: 2023-06-23T01:19:39.576+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:39.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:39 smithi157 conmon[145357]: 2023-06-23T01:19:39.576+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:19:39.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:39 smithi157 conmon[145357]: 2023-06-23T01:19:39.576+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:19:40.010 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:39 smithi120 conmon[117204]: 2023-06-23T01:19:39.681+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:40.011 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:39 smithi120 conmon[117204]: 2023-06-23T01:19:39.681+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:19:40.011 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:39 smithi120 conmon[117204]: 2023-06-23T01:19:39.681+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.597925+0000 front 2023-06-23T01:19:05.597907+0000 (oldest deadline 2023-06-23T01:19:31.497672+0000) 2023-06-23T01:19:40.376 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:39 smithi157 conmon[138256]: 2023-06-23T01:19:39.917+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:40.376 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:39 smithi157 conmon[138256]: 2023-06-23T01:19:39.917+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.766636+0000 front 2023-06-23T01:19:01.766694+0000 (oldest deadline 2023-06-23T01:19:24.666623+0000) 2023-06-23T01:19:40.376 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:39 smithi157 conmon[138256]: 2023-06-23T01:19:39.917+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.167466+0000 front 2023-06-23T01:19:05.167527+0000 (oldest deadline 2023-06-23T01:19:30.467672+0000) 2023-06-23T01:19:40.444 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:39 smithi134 conmon[136657]: 2023-06-23T01:19:39.970+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.513557+0000 front 2023-06-23T01:19:01.513497+0000 (oldest deadline 2023-06-23T01:19:25.013698+0000) 2023-06-23T01:19:40.444 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:39 smithi134 conmon[136657]: 2023-06-23T01:19:39.970+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:09.615111+0000 front 2023-06-23T01:19:09.615136+0000 (oldest deadline 2023-06-23T01:19:34.915349+0000) 2023-06-23T01:19:40.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:39 smithi134 conmon[143713]: 2023-06-23T01:19:39.976+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:40.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:39 smithi134 conmon[143713]: 2023-06-23T01:19:39.976+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:19:40.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:39 smithi134 conmon[143713]: 2023-06-23T01:19:39.976+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:19:40.510 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:40 smithi120 conmon[110061]: 2023-06-23T01:19:40.182+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.689546+0000 front 2023-06-23T01:18:52.689652+0000 (oldest deadline 2023-06-23T01:19:18.589668+0000) 2023-06-23T01:19:40.510 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:40 smithi120 conmon[110061]: 2023-06-23T01:19:40.182+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.590381+0000 front 2023-06-23T01:18:58.590229+0000 (oldest deadline 2023-06-23T01:19:23.290262+0000) 2023-06-23T01:19:40.511 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:40 smithi120 conmon[110061]: 2023-06-23T01:19:40.182+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:09.692142+0000 front 2023-06-23T01:19:09.692161+0000 (oldest deadline 2023-06-23T01:19:32.592217+0000) 2023-06-23T01:19:40.876 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:40 smithi157 conmon[145357]: 2023-06-23T01:19:40.530+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:40.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:40 smithi157 conmon[145357]: 2023-06-23T01:19:40.530+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:19:40.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:40 smithi157 conmon[145357]: 2023-06-23T01:19:40.530+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:19:41.010 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:40 smithi120 conmon[117204]: 2023-06-23T01:19:40.693+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:41.010 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:40 smithi120 conmon[117204]: 2023-06-23T01:19:40.693+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:19:41.011 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:40 smithi120 conmon[117204]: 2023-06-23T01:19:40.693+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.597925+0000 front 2023-06-23T01:19:05.597907+0000 (oldest deadline 2023-06-23T01:19:31.497672+0000) 2023-06-23T01:19:41.011 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:40 smithi120 conmon[117204]: 2023-06-23T01:19:40.693+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.699443+0000 front 2023-06-23T01:19:16.699635+0000 (oldest deadline 2023-06-23T01:19:40.199557+0000) 2023-06-23T01:19:41.194 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:40 smithi134 conmon[136657]: 2023-06-23T01:19:40.939+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.513557+0000 front 2023-06-23T01:19:01.513497+0000 (oldest deadline 2023-06-23T01:19:25.013698+0000) 2023-06-23T01:19:41.194 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:40 smithi134 conmon[136657]: 2023-06-23T01:19:40.939+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:09.615111+0000 front 2023-06-23T01:19:09.615136+0000 (oldest deadline 2023-06-23T01:19:34.915349+0000) 2023-06-23T01:19:41.194 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:40 smithi134 conmon[136657]: 2023-06-23T01:19:40.939+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:14.916128+0000 front 2023-06-23T01:19:14.915946+0000 (oldest deadline 2023-06-23T01:19:40.816027+0000) 2023-06-23T01:19:41.195 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:41 smithi134 conmon[143713]: 2023-06-23T01:19:41.015+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:41.195 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:41 smithi134 conmon[143713]: 2023-06-23T01:19:41.015+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:19:41.195 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:41 smithi134 conmon[143713]: 2023-06-23T01:19:41.015+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:19:41.377 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:40 smithi157 conmon[138256]: 2023-06-23T01:19:40.952+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:41.378 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:40 smithi157 conmon[138256]: 2023-06-23T01:19:40.952+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.766636+0000 front 2023-06-23T01:19:01.766694+0000 (oldest deadline 2023-06-23T01:19:24.666623+0000) 2023-06-23T01:19:41.378 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:40 smithi157 conmon[138256]: 2023-06-23T01:19:40.952+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.167466+0000 front 2023-06-23T01:19:05.167527+0000 (oldest deadline 2023-06-23T01:19:30.467672+0000) 2023-06-23T01:19:41.378 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:40 smithi157 conmon[138256]: 2023-06-23T01:19:40.952+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.770277+0000 front 2023-06-23T01:19:16.770175+0000 (oldest deadline 2023-06-23T01:19:40.270137+0000) 2023-06-23T01:19:41.510 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:41 smithi120 conmon[110061]: 2023-06-23T01:19:41.185+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.689546+0000 front 2023-06-23T01:18:52.689652+0000 (oldest deadline 2023-06-23T01:19:18.589668+0000) 2023-06-23T01:19:41.510 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:41 smithi120 conmon[110061]: 2023-06-23T01:19:41.185+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.590381+0000 front 2023-06-23T01:18:58.590229+0000 (oldest deadline 2023-06-23T01:19:23.290262+0000) 2023-06-23T01:19:41.512 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:41 smithi120 conmon[110061]: 2023-06-23T01:19:41.185+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:09.692142+0000 front 2023-06-23T01:19:09.692161+0000 (oldest deadline 2023-06-23T01:19:32.592217+0000) 2023-06-23T01:19:41.512 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:41 smithi120 conmon[110061]: 2023-06-23T01:19:41.185+0000 7f8bcd17e700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:14.893278+0000 front 2023-06-23T01:19:14.893421+0000 (oldest deadline 2023-06-23T01:19:40.193485+0000) 2023-06-23T01:19:41.876 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:41 smithi157 conmon[145357]: 2023-06-23T01:19:41.558+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:41.876 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:41 smithi157 conmon[145357]: 2023-06-23T01:19:41.558+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:19:41.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:41 smithi157 conmon[145357]: 2023-06-23T01:19:41.558+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:19:42.010 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:41 smithi120 conmon[117204]: 2023-06-23T01:19:41.666+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:42.010 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:41 smithi120 conmon[117204]: 2023-06-23T01:19:41.666+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:19:42.010 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:41 smithi120 conmon[117204]: 2023-06-23T01:19:41.666+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.597925+0000 front 2023-06-23T01:19:05.597907+0000 (oldest deadline 2023-06-23T01:19:31.497672+0000) 2023-06-23T01:19:42.011 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:41 smithi120 conmon[117204]: 2023-06-23T01:19:41.666+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.699443+0000 front 2023-06-23T01:19:16.699635+0000 (oldest deadline 2023-06-23T01:19:40.199557+0000) 2023-06-23T01:19:42.194 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:41 smithi134 conmon[136657]: 2023-06-23T01:19:41.934+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.513557+0000 front 2023-06-23T01:19:01.513497+0000 (oldest deadline 2023-06-23T01:19:25.013698+0000) 2023-06-23T01:19:42.194 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:41 smithi134 conmon[136657]: 2023-06-23T01:19:41.934+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:09.615111+0000 front 2023-06-23T01:19:09.615136+0000 (oldest deadline 2023-06-23T01:19:34.915349+0000) 2023-06-23T01:19:42.194 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:41 smithi134 conmon[136657]: 2023-06-23T01:19:41.934+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:14.916128+0000 front 2023-06-23T01:19:14.915946+0000 (oldest deadline 2023-06-23T01:19:40.816027+0000) 2023-06-23T01:19:42.194 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:42 smithi134 conmon[143713]: 2023-06-23T01:19:42.053+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:42.194 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:42 smithi134 conmon[143713]: 2023-06-23T01:19:42.053+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:19:42.195 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:42 smithi134 conmon[143713]: 2023-06-23T01:19:42.053+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:19:42.376 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:41 smithi157 conmon[138256]: 2023-06-23T01:19:41.920+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:42.376 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:41 smithi157 conmon[138256]: 2023-06-23T01:19:41.920+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.766636+0000 front 2023-06-23T01:19:01.766694+0000 (oldest deadline 2023-06-23T01:19:24.666623+0000) 2023-06-23T01:19:42.377 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:41 smithi157 conmon[138256]: 2023-06-23T01:19:41.920+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.167466+0000 front 2023-06-23T01:19:05.167527+0000 (oldest deadline 2023-06-23T01:19:30.467672+0000) 2023-06-23T01:19:42.377 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:41 smithi157 conmon[138256]: 2023-06-23T01:19:41.920+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.770277+0000 front 2023-06-23T01:19:16.770175+0000 (oldest deadline 2023-06-23T01:19:40.270137+0000) 2023-06-23T01:19:42.876 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:42 smithi157 conmon[145357]: 2023-06-23T01:19:42.601+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:42.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:42 smithi157 conmon[145357]: 2023-06-23T01:19:42.601+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:19:42.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:42 smithi157 conmon[145357]: 2023-06-23T01:19:42.601+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:19:42.900 INFO:journalctl@ceph.osd.6.smithi120.stdout:Jun 23 01:19:42 smithi120 bash[129223]: ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c-osd-6 2023-06-23T01:19:42.901 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:42 smithi120 conmon[117204]: 2023-06-23T01:19:42.644+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:42.901 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:42 smithi120 conmon[117204]: 2023-06-23T01:19:42.644+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:19:42.902 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:42 smithi120 conmon[117204]: 2023-06-23T01:19:42.644+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.597925+0000 front 2023-06-23T01:19:05.597907+0000 (oldest deadline 2023-06-23T01:19:31.497672+0000) 2023-06-23T01:19:42.902 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:42 smithi120 conmon[117204]: 2023-06-23T01:19:42.644+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.699443+0000 front 2023-06-23T01:19:16.699635+0000 (oldest deadline 2023-06-23T01:19:40.199557+0000) 2023-06-23T01:19:43.376 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:42 smithi157 conmon[138256]: 2023-06-23T01:19:42.887+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:43.377 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:42 smithi157 conmon[138256]: 2023-06-23T01:19:42.887+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.766636+0000 front 2023-06-23T01:19:01.766694+0000 (oldest deadline 2023-06-23T01:19:24.666623+0000) 2023-06-23T01:19:43.377 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:42 smithi157 conmon[138256]: 2023-06-23T01:19:42.887+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.167466+0000 front 2023-06-23T01:19:05.167527+0000 (oldest deadline 2023-06-23T01:19:30.467672+0000) 2023-06-23T01:19:43.377 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:42 smithi157 conmon[138256]: 2023-06-23T01:19:42.887+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.770277+0000 front 2023-06-23T01:19:16.770175+0000 (oldest deadline 2023-06-23T01:19:40.270137+0000) 2023-06-23T01:19:43.443 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:42 smithi134 conmon[136657]: 2023-06-23T01:19:42.975+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.513557+0000 front 2023-06-23T01:19:01.513497+0000 (oldest deadline 2023-06-23T01:19:25.013698+0000) 2023-06-23T01:19:43.444 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:42 smithi134 conmon[136657]: 2023-06-23T01:19:42.975+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:09.615111+0000 front 2023-06-23T01:19:09.615136+0000 (oldest deadline 2023-06-23T01:19:34.915349+0000) 2023-06-23T01:19:43.444 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:42 smithi134 conmon[136657]: 2023-06-23T01:19:42.975+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:14.916128+0000 front 2023-06-23T01:19:14.915946+0000 (oldest deadline 2023-06-23T01:19:40.816027+0000) 2023-06-23T01:19:43.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:43 smithi134 conmon[143713]: 2023-06-23T01:19:43.054+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:43.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:43 smithi134 conmon[143713]: 2023-06-23T01:19:43.054+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:19:43.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:43 smithi134 conmon[143713]: 2023-06-23T01:19:43.054+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:19:43.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:43 smithi134 conmon[143713]: 2023-06-23T01:19:43.054+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:17.389622+0000 front 2023-06-23T01:19:17.389746+0000 (oldest deadline 2023-06-23T01:19:42.089671+0000) 2023-06-23T01:19:43.903 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:43 smithi157 conmon[145357]: 2023-06-23T01:19:43.646+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:43.904 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:43 smithi157 conmon[145357]: 2023-06-23T01:19:43.646+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:19:43.904 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:43 smithi157 conmon[145357]: 2023-06-23T01:19:43.646+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:19:44.010 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:43 smithi120 conmon[117204]: 2023-06-23T01:19:43.672+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:44.011 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:43 smithi120 conmon[117204]: 2023-06-23T01:19:43.672+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:19:44.011 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:43 smithi120 conmon[117204]: 2023-06-23T01:19:43.672+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.597925+0000 front 2023-06-23T01:19:05.597907+0000 (oldest deadline 2023-06-23T01:19:31.497672+0000) 2023-06-23T01:19:44.011 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:43 smithi120 conmon[117204]: 2023-06-23T01:19:43.672+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.699443+0000 front 2023-06-23T01:19:16.699635+0000 (oldest deadline 2023-06-23T01:19:40.199557+0000) 2023-06-23T01:19:44.376 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:43 smithi157 conmon[138256]: 2023-06-23T01:19:43.900+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:44.377 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:43 smithi157 conmon[138256]: 2023-06-23T01:19:43.900+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.766636+0000 front 2023-06-23T01:19:01.766694+0000 (oldest deadline 2023-06-23T01:19:24.666623+0000) 2023-06-23T01:19:44.377 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:43 smithi157 conmon[138256]: 2023-06-23T01:19:43.900+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.167466+0000 front 2023-06-23T01:19:05.167527+0000 (oldest deadline 2023-06-23T01:19:30.467672+0000) 2023-06-23T01:19:44.377 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:43 smithi157 conmon[138256]: 2023-06-23T01:19:43.900+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.770277+0000 front 2023-06-23T01:19:16.770175+0000 (oldest deadline 2023-06-23T01:19:40.270137+0000) 2023-06-23T01:19:44.444 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:43 smithi134 conmon[136657]: 2023-06-23T01:19:43.946+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.513557+0000 front 2023-06-23T01:19:01.513497+0000 (oldest deadline 2023-06-23T01:19:25.013698+0000) 2023-06-23T01:19:44.444 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:43 smithi134 conmon[136657]: 2023-06-23T01:19:43.946+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:09.615111+0000 front 2023-06-23T01:19:09.615136+0000 (oldest deadline 2023-06-23T01:19:34.915349+0000) 2023-06-23T01:19:44.444 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:43 smithi134 conmon[136657]: 2023-06-23T01:19:43.946+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:14.916128+0000 front 2023-06-23T01:19:14.915946+0000 (oldest deadline 2023-06-23T01:19:40.816027+0000) 2023-06-23T01:19:44.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:44 smithi134 conmon[143713]: 2023-06-23T01:19:44.037+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:44.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:44 smithi134 conmon[143713]: 2023-06-23T01:19:44.037+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:19:44.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:44 smithi134 conmon[143713]: 2023-06-23T01:19:44.037+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:19:44.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:44 smithi134 conmon[143713]: 2023-06-23T01:19:44.037+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:17.389622+0000 front 2023-06-23T01:19:17.389746+0000 (oldest deadline 2023-06-23T01:19:42.089671+0000) 2023-06-23T01:19:44.739 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@osd.6.service' 2023-06-23T01:19:44.821 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T01:19:44.821 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-06-23T01:19:44.822 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-06-23T01:19:44.822 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@osd.7 2023-06-23T01:19:44.940 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:44 smithi157 conmon[145357]: 2023-06-23T01:19:44.632+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:44.940 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:44 smithi157 conmon[145357]: 2023-06-23T01:19:44.632+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:19:44.940 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:44 smithi157 conmon[145357]: 2023-06-23T01:19:44.632+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:19:45.010 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:44 smithi120 conmon[117204]: 2023-06-23T01:19:44.690+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:45.010 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:44 smithi120 conmon[117204]: 2023-06-23T01:19:44.690+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:19:45.010 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:44 smithi120 conmon[117204]: 2023-06-23T01:19:44.690+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.597925+0000 front 2023-06-23T01:19:05.597907+0000 (oldest deadline 2023-06-23T01:19:31.497672+0000) 2023-06-23T01:19:45.011 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:44 smithi120 conmon[117204]: 2023-06-23T01:19:44.690+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.699443+0000 front 2023-06-23T01:19:16.699635+0000 (oldest deadline 2023-06-23T01:19:40.199557+0000) 2023-06-23T01:19:45.174 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:45 smithi134 conmon[143713]: 2023-06-23T01:19:45.054+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:45.174 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:45 smithi134 conmon[143713]: 2023-06-23T01:19:45.055+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:19:45.175 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:45 smithi134 conmon[143713]: 2023-06-23T01:19:45.055+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:19:45.175 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:45 smithi134 conmon[143713]: 2023-06-23T01:19:45.055+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:17.389622+0000 front 2023-06-23T01:19:17.389746+0000 (oldest deadline 2023-06-23T01:19:42.089671+0000) 2023-06-23T01:19:45.175 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:44 smithi134 systemd[1]: Stopping Ceph osd.7 for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:19:45.175 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:44 smithi134 conmon[136657]: 2023-06-23T01:19:44.970+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.513557+0000 front 2023-06-23T01:19:01.513497+0000 (oldest deadline 2023-06-23T01:19:25.013698+0000) 2023-06-23T01:19:45.175 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:44 smithi134 conmon[136657]: 2023-06-23T01:19:44.970+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:09.615111+0000 front 2023-06-23T01:19:09.615136+0000 (oldest deadline 2023-06-23T01:19:34.915349+0000) 2023-06-23T01:19:45.176 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:44 smithi134 conmon[136657]: 2023-06-23T01:19:44.970+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:14.916128+0000 front 2023-06-23T01:19:14.915946+0000 (oldest deadline 2023-06-23T01:19:40.816027+0000) 2023-06-23T01:19:45.176 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:45 smithi134 conmon[136657]: 2023-06-23T01:19:45.171+0000 7fddf09bb700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T01:19:45.176 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:45 smithi134 conmon[136657]: 2023-06-23T01:19:45.171+0000 7fddf09bb700 -1 osd.7 85 *** Got signal Terminated *** 2023-06-23T01:19:45.379 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:44 smithi157 conmon[138256]: 2023-06-23T01:19:44.937+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:45.379 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:44 smithi157 conmon[138256]: 2023-06-23T01:19:44.937+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.766636+0000 front 2023-06-23T01:19:01.766694+0000 (oldest deadline 2023-06-23T01:19:24.666623+0000) 2023-06-23T01:19:45.379 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:44 smithi157 conmon[138256]: 2023-06-23T01:19:44.937+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.167466+0000 front 2023-06-23T01:19:05.167527+0000 (oldest deadline 2023-06-23T01:19:30.467672+0000) 2023-06-23T01:19:45.379 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:44 smithi157 conmon[138256]: 2023-06-23T01:19:44.937+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.770277+0000 front 2023-06-23T01:19:16.770175+0000 (oldest deadline 2023-06-23T01:19:40.270137+0000) 2023-06-23T01:19:45.443 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:45 smithi134 conmon[136657]: 2023-06-23T01:19:45.171+0000 7fddf09bb700 -1 osd.7 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-23T01:19:45.973 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:45 smithi157 conmon[145357]: 2023-06-23T01:19:45.663+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:45.974 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:45 smithi157 conmon[145357]: 2023-06-23T01:19:45.663+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:19:45.974 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:45 smithi157 conmon[145357]: 2023-06-23T01:19:45.663+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:19:46.010 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:45 smithi120 conmon[117204]: 2023-06-23T01:19:45.665+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:46.010 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:45 smithi120 conmon[117204]: 2023-06-23T01:19:45.665+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:19:46.011 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:45 smithi120 conmon[117204]: 2023-06-23T01:19:45.665+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.597925+0000 front 2023-06-23T01:19:05.597907+0000 (oldest deadline 2023-06-23T01:19:31.497672+0000) 2023-06-23T01:19:46.011 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:45 smithi120 conmon[117204]: 2023-06-23T01:19:45.665+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.699443+0000 front 2023-06-23T01:19:16.699635+0000 (oldest deadline 2023-06-23T01:19:40.199557+0000) 2023-06-23T01:19:46.194 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:45 smithi134 conmon[136657]: 2023-06-23T01:19:45.926+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.513557+0000 front 2023-06-23T01:19:01.513497+0000 (oldest deadline 2023-06-23T01:19:25.013698+0000) 2023-06-23T01:19:46.194 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:45 smithi134 conmon[136657]: 2023-06-23T01:19:45.926+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:09.615111+0000 front 2023-06-23T01:19:09.615136+0000 (oldest deadline 2023-06-23T01:19:34.915349+0000) 2023-06-23T01:19:46.194 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:45 smithi134 conmon[136657]: 2023-06-23T01:19:45.926+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:14.916128+0000 front 2023-06-23T01:19:14.915946+0000 (oldest deadline 2023-06-23T01:19:40.816027+0000) 2023-06-23T01:19:46.195 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:46 smithi134 conmon[143713]: 2023-06-23T01:19:46.056+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:46.195 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:46 smithi134 conmon[143713]: 2023-06-23T01:19:46.056+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:19:46.195 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:46 smithi134 conmon[143713]: 2023-06-23T01:19:46.056+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:19:46.195 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:46 smithi134 conmon[143713]: 2023-06-23T01:19:46.056+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:17.389622+0000 front 2023-06-23T01:19:17.389746+0000 (oldest deadline 2023-06-23T01:19:42.089671+0000) 2023-06-23T01:19:46.376 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:45 smithi157 conmon[138256]: 2023-06-23T01:19:45.970+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:46.377 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:45 smithi157 conmon[138256]: 2023-06-23T01:19:45.970+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.766636+0000 front 2023-06-23T01:19:01.766694+0000 (oldest deadline 2023-06-23T01:19:24.666623+0000) 2023-06-23T01:19:46.377 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:45 smithi157 conmon[138256]: 2023-06-23T01:19:45.970+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.167466+0000 front 2023-06-23T01:19:05.167527+0000 (oldest deadline 2023-06-23T01:19:30.467672+0000) 2023-06-23T01:19:46.377 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:45 smithi157 conmon[138256]: 2023-06-23T01:19:45.970+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.770277+0000 front 2023-06-23T01:19:16.770175+0000 (oldest deadline 2023-06-23T01:19:40.270137+0000) 2023-06-23T01:19:46.927 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:46 smithi157 conmon[145357]: 2023-06-23T01:19:46.673+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:46.928 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:46 smithi157 conmon[145357]: 2023-06-23T01:19:46.673+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:19:46.928 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:46 smithi157 conmon[145357]: 2023-06-23T01:19:46.673+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:19:47.010 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:46 smithi120 conmon[117204]: 2023-06-23T01:19:46.634+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:47.010 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:46 smithi120 conmon[117204]: 2023-06-23T01:19:46.634+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:19:47.010 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:46 smithi120 conmon[117204]: 2023-06-23T01:19:46.634+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.597925+0000 front 2023-06-23T01:19:05.597907+0000 (oldest deadline 2023-06-23T01:19:31.497672+0000) 2023-06-23T01:19:47.011 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:46 smithi120 conmon[117204]: 2023-06-23T01:19:46.634+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.699443+0000 front 2023-06-23T01:19:16.699635+0000 (oldest deadline 2023-06-23T01:19:40.199557+0000) 2023-06-23T01:19:47.011 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:46 smithi120 conmon[117204]: 2023-06-23T01:19:46.634+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.500686+0000 front 2023-06-23T01:19:25.500560+0000 (oldest deadline 2023-06-23T01:19:46.600427+0000) 2023-06-23T01:19:47.194 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:46 smithi134 conmon[136657]: 2023-06-23T01:19:46.933+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.513557+0000 front 2023-06-23T01:19:01.513497+0000 (oldest deadline 2023-06-23T01:19:25.013698+0000) 2023-06-23T01:19:47.194 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:46 smithi134 conmon[136657]: 2023-06-23T01:19:46.934+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:09.615111+0000 front 2023-06-23T01:19:09.615136+0000 (oldest deadline 2023-06-23T01:19:34.915349+0000) 2023-06-23T01:19:47.194 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:46 smithi134 conmon[136657]: 2023-06-23T01:19:46.934+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:14.916128+0000 front 2023-06-23T01:19:14.915946+0000 (oldest deadline 2023-06-23T01:19:40.816027+0000) 2023-06-23T01:19:47.195 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:47 smithi134 conmon[143713]: 2023-06-23T01:19:47.069+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:47.195 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:47 smithi134 conmon[143713]: 2023-06-23T01:19:47.069+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:19:47.195 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:47 smithi134 conmon[143713]: 2023-06-23T01:19:47.069+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:19:47.195 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:47 smithi134 conmon[143713]: 2023-06-23T01:19:47.069+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:17.389622+0000 front 2023-06-23T01:19:17.389746+0000 (oldest deadline 2023-06-23T01:19:42.089671+0000) 2023-06-23T01:19:47.376 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:46 smithi157 conmon[138256]: 2023-06-23T01:19:46.924+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:47.376 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:46 smithi157 conmon[138256]: 2023-06-23T01:19:46.924+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.766636+0000 front 2023-06-23T01:19:01.766694+0000 (oldest deadline 2023-06-23T01:19:24.666623+0000) 2023-06-23T01:19:47.377 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:46 smithi157 conmon[138256]: 2023-06-23T01:19:46.924+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.167466+0000 front 2023-06-23T01:19:05.167527+0000 (oldest deadline 2023-06-23T01:19:30.467672+0000) 2023-06-23T01:19:47.377 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:46 smithi157 conmon[138256]: 2023-06-23T01:19:46.924+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.770277+0000 front 2023-06-23T01:19:16.770175+0000 (oldest deadline 2023-06-23T01:19:40.270137+0000) 2023-06-23T01:19:47.927 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:47 smithi157 conmon[145357]: 2023-06-23T01:19:47.666+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:47.927 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:47 smithi157 conmon[145357]: 2023-06-23T01:19:47.666+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:19:47.927 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:47 smithi157 conmon[145357]: 2023-06-23T01:19:47.666+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:19:48.010 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:47 smithi120 conmon[117204]: 2023-06-23T01:19:47.661+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:48.010 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:47 smithi120 conmon[117204]: 2023-06-23T01:19:47.661+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:19:48.010 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:47 smithi120 conmon[117204]: 2023-06-23T01:19:47.661+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.597925+0000 front 2023-06-23T01:19:05.597907+0000 (oldest deadline 2023-06-23T01:19:31.497672+0000) 2023-06-23T01:19:48.011 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:47 smithi120 conmon[117204]: 2023-06-23T01:19:47.661+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.699443+0000 front 2023-06-23T01:19:16.699635+0000 (oldest deadline 2023-06-23T01:19:40.199557+0000) 2023-06-23T01:19:48.011 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:47 smithi120 conmon[117204]: 2023-06-23T01:19:47.661+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.500686+0000 front 2023-06-23T01:19:25.500560+0000 (oldest deadline 2023-06-23T01:19:46.600427+0000) 2023-06-23T01:19:48.194 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:47 smithi134 conmon[136657]: 2023-06-23T01:19:47.888+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.513557+0000 front 2023-06-23T01:19:01.513497+0000 (oldest deadline 2023-06-23T01:19:25.013698+0000) 2023-06-23T01:19:48.194 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:47 smithi134 conmon[136657]: 2023-06-23T01:19:47.888+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:09.615111+0000 front 2023-06-23T01:19:09.615136+0000 (oldest deadline 2023-06-23T01:19:34.915349+0000) 2023-06-23T01:19:48.194 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:47 smithi134 conmon[136657]: 2023-06-23T01:19:47.888+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:14.916128+0000 front 2023-06-23T01:19:14.915946+0000 (oldest deadline 2023-06-23T01:19:40.816027+0000) 2023-06-23T01:19:48.194 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:48 smithi134 conmon[143713]: 2023-06-23T01:19:48.074+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:48.195 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:48 smithi134 conmon[143713]: 2023-06-23T01:19:48.074+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:19:48.195 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:48 smithi134 conmon[143713]: 2023-06-23T01:19:48.074+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:19:48.195 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:48 smithi134 conmon[143713]: 2023-06-23T01:19:48.074+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:17.389622+0000 front 2023-06-23T01:19:17.389746+0000 (oldest deadline 2023-06-23T01:19:42.089671+0000) 2023-06-23T01:19:48.195 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:48 smithi134 conmon[143713]: 2023-06-23T01:19:48.075+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:22.090003+0000 front 2023-06-23T01:19:22.090048+0000 (oldest deadline 2023-06-23T01:19:47.390149+0000) 2023-06-23T01:19:48.376 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:47 smithi157 conmon[138256]: 2023-06-23T01:19:47.924+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:48.377 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:47 smithi157 conmon[138256]: 2023-06-23T01:19:47.924+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.766636+0000 front 2023-06-23T01:19:01.766694+0000 (oldest deadline 2023-06-23T01:19:24.666623+0000) 2023-06-23T01:19:48.377 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:47 smithi157 conmon[138256]: 2023-06-23T01:19:47.924+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.167466+0000 front 2023-06-23T01:19:05.167527+0000 (oldest deadline 2023-06-23T01:19:30.467672+0000) 2023-06-23T01:19:48.377 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:47 smithi157 conmon[138256]: 2023-06-23T01:19:47.924+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.770277+0000 front 2023-06-23T01:19:16.770175+0000 (oldest deadline 2023-06-23T01:19:40.270137+0000) 2023-06-23T01:19:49.010 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:48 smithi120 conmon[117204]: 2023-06-23T01:19:48.700+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:49.010 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:48 smithi120 conmon[117204]: 2023-06-23T01:19:48.700+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:19:49.010 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:48 smithi120 conmon[117204]: 2023-06-23T01:19:48.700+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.597925+0000 front 2023-06-23T01:19:05.597907+0000 (oldest deadline 2023-06-23T01:19:31.497672+0000) 2023-06-23T01:19:49.010 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:48 smithi120 conmon[117204]: 2023-06-23T01:19:48.700+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.699443+0000 front 2023-06-23T01:19:16.699635+0000 (oldest deadline 2023-06-23T01:19:40.199557+0000) 2023-06-23T01:19:49.011 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:48 smithi120 conmon[117204]: 2023-06-23T01:19:48.700+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.500686+0000 front 2023-06-23T01:19:25.500560+0000 (oldest deadline 2023-06-23T01:19:46.600427+0000) 2023-06-23T01:19:49.126 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:48 smithi157 conmon[138256]: 2023-06-23T01:19:48.917+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:49.127 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:48 smithi157 conmon[138256]: 2023-06-23T01:19:48.917+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.766636+0000 front 2023-06-23T01:19:01.766694+0000 (oldest deadline 2023-06-23T01:19:24.666623+0000) 2023-06-23T01:19:49.127 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:48 smithi157 conmon[138256]: 2023-06-23T01:19:48.917+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.167466+0000 front 2023-06-23T01:19:05.167527+0000 (oldest deadline 2023-06-23T01:19:30.467672+0000) 2023-06-23T01:19:49.127 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:48 smithi157 conmon[138256]: 2023-06-23T01:19:48.917+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.770277+0000 front 2023-06-23T01:19:16.770175+0000 (oldest deadline 2023-06-23T01:19:40.270137+0000) 2023-06-23T01:19:49.127 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:48 smithi157 conmon[138256]: 2023-06-23T01:19:48.917+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:22.571301+0000 front 2023-06-23T01:19:22.571233+0000 (oldest deadline 2023-06-23T01:19:48.471321+0000) 2023-06-23T01:19:49.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:48 smithi157 conmon[145357]: 2023-06-23T01:19:48.668+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:49.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:48 smithi157 conmon[145357]: 2023-06-23T01:19:48.668+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:19:49.128 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:48 smithi157 conmon[145357]: 2023-06-23T01:19:48.668+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:19:49.194 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:48 smithi134 conmon[136657]: 2023-06-23T01:19:48.888+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.513557+0000 front 2023-06-23T01:19:01.513497+0000 (oldest deadline 2023-06-23T01:19:25.013698+0000) 2023-06-23T01:19:49.194 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:48 smithi134 conmon[136657]: 2023-06-23T01:19:48.888+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:09.615111+0000 front 2023-06-23T01:19:09.615136+0000 (oldest deadline 2023-06-23T01:19:34.915349+0000) 2023-06-23T01:19:49.194 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:48 smithi134 conmon[136657]: 2023-06-23T01:19:48.888+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:14.916128+0000 front 2023-06-23T01:19:14.915946+0000 (oldest deadline 2023-06-23T01:19:40.816027+0000) 2023-06-23T01:19:49.194 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:48 smithi134 conmon[136657]: 2023-06-23T01:19:48.888+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:20.816496+0000 front 2023-06-23T01:19:20.816395+0000 (oldest deadline 2023-06-23T01:19:48.417174+0000) 2023-06-23T01:19:49.195 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:49 smithi134 conmon[143713]: 2023-06-23T01:19:49.028+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:49.195 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:49 smithi134 conmon[143713]: 2023-06-23T01:19:49.028+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:19:49.195 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:49 smithi134 conmon[143713]: 2023-06-23T01:19:49.028+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:19:49.195 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:49 smithi134 conmon[143713]: 2023-06-23T01:19:49.028+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:17.389622+0000 front 2023-06-23T01:19:17.389746+0000 (oldest deadline 2023-06-23T01:19:42.089671+0000) 2023-06-23T01:19:49.195 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:49 smithi134 conmon[143713]: 2023-06-23T01:19:49.028+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:22.090003+0000 front 2023-06-23T01:19:22.090048+0000 (oldest deadline 2023-06-23T01:19:47.390149+0000) 2023-06-23T01:19:50.010 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:49 smithi120 conmon[117204]: 2023-06-23T01:19:49.720+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:50.011 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:49 smithi120 conmon[117204]: 2023-06-23T01:19:49.720+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:19:50.011 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:49 smithi120 conmon[117204]: 2023-06-23T01:19:49.720+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.597925+0000 front 2023-06-23T01:19:05.597907+0000 (oldest deadline 2023-06-23T01:19:31.497672+0000) 2023-06-23T01:19:50.011 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:49 smithi120 conmon[117204]: 2023-06-23T01:19:49.720+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.699443+0000 front 2023-06-23T01:19:16.699635+0000 (oldest deadline 2023-06-23T01:19:40.199557+0000) 2023-06-23T01:19:50.011 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:49 smithi120 conmon[117204]: 2023-06-23T01:19:49.720+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.500686+0000 front 2023-06-23T01:19:25.500560+0000 (oldest deadline 2023-06-23T01:19:46.600427+0000) 2023-06-23T01:19:50.126 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:49 smithi157 conmon[145357]: 2023-06-23T01:19:49.665+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:50.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:49 smithi157 conmon[145357]: 2023-06-23T01:19:49.665+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:19:50.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:49 smithi157 conmon[145357]: 2023-06-23T01:19:49.665+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:19:50.127 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:49 smithi157 conmon[138256]: 2023-06-23T01:19:49.904+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:50.127 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:49 smithi157 conmon[138256]: 2023-06-23T01:19:49.904+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.766636+0000 front 2023-06-23T01:19:01.766694+0000 (oldest deadline 2023-06-23T01:19:24.666623+0000) 2023-06-23T01:19:50.128 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:49 smithi157 conmon[138256]: 2023-06-23T01:19:49.904+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.167466+0000 front 2023-06-23T01:19:05.167527+0000 (oldest deadline 2023-06-23T01:19:30.467672+0000) 2023-06-23T01:19:50.128 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:49 smithi157 conmon[138256]: 2023-06-23T01:19:49.904+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.770277+0000 front 2023-06-23T01:19:16.770175+0000 (oldest deadline 2023-06-23T01:19:40.270137+0000) 2023-06-23T01:19:50.128 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:49 smithi157 conmon[138256]: 2023-06-23T01:19:49.904+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:22.571301+0000 front 2023-06-23T01:19:22.571233+0000 (oldest deadline 2023-06-23T01:19:48.471321+0000) 2023-06-23T01:19:50.194 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:49 smithi134 conmon[136657]: 2023-06-23T01:19:49.865+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.513557+0000 front 2023-06-23T01:19:01.513497+0000 (oldest deadline 2023-06-23T01:19:25.013698+0000) 2023-06-23T01:19:50.196 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:49 smithi134 conmon[136657]: 2023-06-23T01:19:49.865+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:09.615111+0000 front 2023-06-23T01:19:09.615136+0000 (oldest deadline 2023-06-23T01:19:34.915349+0000) 2023-06-23T01:19:50.196 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:49 smithi134 conmon[136657]: 2023-06-23T01:19:49.865+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:14.916128+0000 front 2023-06-23T01:19:14.915946+0000 (oldest deadline 2023-06-23T01:19:40.816027+0000) 2023-06-23T01:19:50.196 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:49 smithi134 conmon[136657]: 2023-06-23T01:19:49.865+0000 7fdde9fb3700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:20.816496+0000 front 2023-06-23T01:19:20.816395+0000 (oldest deadline 2023-06-23T01:19:48.417174+0000) 2023-06-23T01:19:50.196 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:50 smithi134 conmon[143713]: 2023-06-23T01:19:50.060+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:50.197 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:50 smithi134 conmon[143713]: 2023-06-23T01:19:50.060+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:19:50.197 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:50 smithi134 conmon[143713]: 2023-06-23T01:19:50.060+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:19:50.197 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:50 smithi134 conmon[143713]: 2023-06-23T01:19:50.060+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:17.389622+0000 front 2023-06-23T01:19:17.389746+0000 (oldest deadline 2023-06-23T01:19:42.089671+0000) 2023-06-23T01:19:50.197 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:50 smithi134 conmon[143713]: 2023-06-23T01:19:50.060+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:22.090003+0000 front 2023-06-23T01:19:22.090048+0000 (oldest deadline 2023-06-23T01:19:47.390149+0000) 2023-06-23T01:19:51.010 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:50 smithi120 conmon[117204]: 2023-06-23T01:19:50.746+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:51.010 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:50 smithi120 conmon[117204]: 2023-06-23T01:19:50.746+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:19:51.011 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:50 smithi120 conmon[117204]: 2023-06-23T01:19:50.746+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.597925+0000 front 2023-06-23T01:19:05.597907+0000 (oldest deadline 2023-06-23T01:19:31.497672+0000) 2023-06-23T01:19:51.011 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:50 smithi120 conmon[117204]: 2023-06-23T01:19:50.746+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.699443+0000 front 2023-06-23T01:19:16.699635+0000 (oldest deadline 2023-06-23T01:19:40.199557+0000) 2023-06-23T01:19:51.011 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:50 smithi120 conmon[117204]: 2023-06-23T01:19:50.746+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.500686+0000 front 2023-06-23T01:19:25.500560+0000 (oldest deadline 2023-06-23T01:19:46.600427+0000) 2023-06-23T01:19:51.025 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jun 23 01:19:50 smithi134 bash[152921]: ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c-osd-7 2023-06-23T01:19:51.126 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:50 smithi157 conmon[145357]: 2023-06-23T01:19:50.686+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:51.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:50 smithi157 conmon[145357]: 2023-06-23T01:19:50.686+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:19:51.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:50 smithi157 conmon[145357]: 2023-06-23T01:19:50.686+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:19:51.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:50 smithi157 conmon[145357]: 2023-06-23T01:19:50.686+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.749536+0000 front 2023-06-23T01:19:25.749472+0000 (oldest deadline 2023-06-23T01:19:49.849573+0000) 2023-06-23T01:19:51.127 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:50 smithi157 conmon[138256]: 2023-06-23T01:19:50.893+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:51.127 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:50 smithi157 conmon[138256]: 2023-06-23T01:19:50.893+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.766636+0000 front 2023-06-23T01:19:01.766694+0000 (oldest deadline 2023-06-23T01:19:24.666623+0000) 2023-06-23T01:19:51.128 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:50 smithi157 conmon[138256]: 2023-06-23T01:19:50.893+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.167466+0000 front 2023-06-23T01:19:05.167527+0000 (oldest deadline 2023-06-23T01:19:30.467672+0000) 2023-06-23T01:19:51.128 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:50 smithi157 conmon[138256]: 2023-06-23T01:19:50.893+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.770277+0000 front 2023-06-23T01:19:16.770175+0000 (oldest deadline 2023-06-23T01:19:40.270137+0000) 2023-06-23T01:19:51.128 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:50 smithi157 conmon[138256]: 2023-06-23T01:19:50.893+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:22.571301+0000 front 2023-06-23T01:19:22.571233+0000 (oldest deadline 2023-06-23T01:19:48.471321+0000) 2023-06-23T01:19:51.443 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:51 smithi134 conmon[143713]: 2023-06-23T01:19:51.022+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:51.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:51 smithi134 conmon[143713]: 2023-06-23T01:19:51.022+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:19:51.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:51 smithi134 conmon[143713]: 2023-06-23T01:19:51.022+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:19:51.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:51 smithi134 conmon[143713]: 2023-06-23T01:19:51.022+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:17.389622+0000 front 2023-06-23T01:19:17.389746+0000 (oldest deadline 2023-06-23T01:19:42.089671+0000) 2023-06-23T01:19:51.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:51 smithi134 conmon[143713]: 2023-06-23T01:19:51.022+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:22.090003+0000 front 2023-06-23T01:19:22.090048+0000 (oldest deadline 2023-06-23T01:19:47.390149+0000) 2023-06-23T01:19:52.126 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:51 smithi157 conmon[145357]: 2023-06-23T01:19:51.682+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:52.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:51 smithi157 conmon[145357]: 2023-06-23T01:19:51.682+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:19:52.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:51 smithi157 conmon[145357]: 2023-06-23T01:19:51.682+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:19:52.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:51 smithi157 conmon[145357]: 2023-06-23T01:19:51.682+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.749536+0000 front 2023-06-23T01:19:25.749472+0000 (oldest deadline 2023-06-23T01:19:49.849573+0000) 2023-06-23T01:19:52.127 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:51 smithi157 conmon[138256]: 2023-06-23T01:19:51.880+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:52.127 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:51 smithi157 conmon[138256]: 2023-06-23T01:19:51.880+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.766636+0000 front 2023-06-23T01:19:01.766694+0000 (oldest deadline 2023-06-23T01:19:24.666623+0000) 2023-06-23T01:19:52.127 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:51 smithi157 conmon[138256]: 2023-06-23T01:19:51.880+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.167466+0000 front 2023-06-23T01:19:05.167527+0000 (oldest deadline 2023-06-23T01:19:30.467672+0000) 2023-06-23T01:19:52.128 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:51 smithi157 conmon[138256]: 2023-06-23T01:19:51.880+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.770277+0000 front 2023-06-23T01:19:16.770175+0000 (oldest deadline 2023-06-23T01:19:40.270137+0000) 2023-06-23T01:19:52.128 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:51 smithi157 conmon[138256]: 2023-06-23T01:19:51.880+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:22.571301+0000 front 2023-06-23T01:19:22.571233+0000 (oldest deadline 2023-06-23T01:19:48.471321+0000) 2023-06-23T01:19:52.260 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:51 smithi120 conmon[117204]: 2023-06-23T01:19:51.778+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:52.260 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:51 smithi120 conmon[117204]: 2023-06-23T01:19:51.778+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:19:52.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:51 smithi120 conmon[117204]: 2023-06-23T01:19:51.778+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.597925+0000 front 2023-06-23T01:19:05.597907+0000 (oldest deadline 2023-06-23T01:19:31.497672+0000) 2023-06-23T01:19:52.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:51 smithi120 conmon[117204]: 2023-06-23T01:19:51.778+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.699443+0000 front 2023-06-23T01:19:16.699635+0000 (oldest deadline 2023-06-23T01:19:40.199557+0000) 2023-06-23T01:19:52.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:51 smithi120 conmon[117204]: 2023-06-23T01:19:51.778+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.500686+0000 front 2023-06-23T01:19:25.500560+0000 (oldest deadline 2023-06-23T01:19:46.600427+0000) 2023-06-23T01:19:52.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:52 smithi134 conmon[143713]: 2023-06-23T01:19:52.052+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:52.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:52 smithi134 conmon[143713]: 2023-06-23T01:19:52.052+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:19:52.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:52 smithi134 conmon[143713]: 2023-06-23T01:19:52.052+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:19:52.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:52 smithi134 conmon[143713]: 2023-06-23T01:19:52.052+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:17.389622+0000 front 2023-06-23T01:19:17.389746+0000 (oldest deadline 2023-06-23T01:19:42.089671+0000) 2023-06-23T01:19:52.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:52 smithi134 conmon[143713]: 2023-06-23T01:19:52.052+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:22.090003+0000 front 2023-06-23T01:19:22.090048+0000 (oldest deadline 2023-06-23T01:19:47.390149+0000) 2023-06-23T01:19:52.526 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@osd.7.service' 2023-06-23T01:19:52.574 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T01:19:52.575 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-06-23T01:19:52.575 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-06-23T01:19:52.575 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@osd.8 2023-06-23T01:19:52.876 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:52 smithi157 conmon[145357]: 2023-06-23T01:19:52.702+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:52.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:52 smithi157 conmon[145357]: 2023-06-23T01:19:52.702+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:19:52.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:52 smithi157 conmon[145357]: 2023-06-23T01:19:52.702+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:19:52.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:52 smithi157 conmon[145357]: 2023-06-23T01:19:52.702+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.749536+0000 front 2023-06-23T01:19:25.749472+0000 (oldest deadline 2023-06-23T01:19:49.849573+0000) 2023-06-23T01:19:52.877 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:52 smithi157 systemd[1]: Stopping Ceph osd.8 for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:19:52.878 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:52 smithi157 conmon[138256]: 2023-06-23T01:19:52.842+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:52.878 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:52 smithi157 conmon[138256]: 2023-06-23T01:19:52.842+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.766636+0000 front 2023-06-23T01:19:01.766694+0000 (oldest deadline 2023-06-23T01:19:24.666623+0000) 2023-06-23T01:19:52.878 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:52 smithi157 conmon[138256]: 2023-06-23T01:19:52.842+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.167466+0000 front 2023-06-23T01:19:05.167527+0000 (oldest deadline 2023-06-23T01:19:30.467672+0000) 2023-06-23T01:19:52.878 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:52 smithi157 conmon[138256]: 2023-06-23T01:19:52.842+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.770277+0000 front 2023-06-23T01:19:16.770175+0000 (oldest deadline 2023-06-23T01:19:40.270137+0000) 2023-06-23T01:19:52.878 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:52 smithi157 conmon[138256]: 2023-06-23T01:19:52.842+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:22.571301+0000 front 2023-06-23T01:19:22.571233+0000 (oldest deadline 2023-06-23T01:19:48.471321+0000) 2023-06-23T01:19:53.260 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:52 smithi120 conmon[117204]: 2023-06-23T01:19:52.818+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:53.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:52 smithi120 conmon[117204]: 2023-06-23T01:19:52.818+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:19:53.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:52 smithi120 conmon[117204]: 2023-06-23T01:19:52.818+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.597925+0000 front 2023-06-23T01:19:05.597907+0000 (oldest deadline 2023-06-23T01:19:31.497672+0000) 2023-06-23T01:19:53.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:52 smithi120 conmon[117204]: 2023-06-23T01:19:52.818+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.699443+0000 front 2023-06-23T01:19:16.699635+0000 (oldest deadline 2023-06-23T01:19:40.199557+0000) 2023-06-23T01:19:53.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:52 smithi120 conmon[117204]: 2023-06-23T01:19:52.818+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.500686+0000 front 2023-06-23T01:19:25.500560+0000 (oldest deadline 2023-06-23T01:19:46.600427+0000) 2023-06-23T01:19:53.376 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:52 smithi157 conmon[138256]: 2023-06-23T01:19:52.934+0000 7fd2e8e49700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T01:19:53.377 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:52 smithi157 conmon[138256]: 2023-06-23T01:19:52.934+0000 7fd2e8e49700 -1 osd.8 85 *** Got signal Terminated *** 2023-06-23T01:19:53.377 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:52 smithi157 conmon[138256]: 2023-06-23T01:19:52.934+0000 7fd2e8e49700 -1 osd.8 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-23T01:19:53.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:53 smithi134 conmon[143713]: 2023-06-23T01:19:53.055+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:53.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:53 smithi134 conmon[143713]: 2023-06-23T01:19:53.055+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:19:53.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:53 smithi134 conmon[143713]: 2023-06-23T01:19:53.055+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:19:53.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:53 smithi134 conmon[143713]: 2023-06-23T01:19:53.055+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:17.389622+0000 front 2023-06-23T01:19:17.389746+0000 (oldest deadline 2023-06-23T01:19:42.089671+0000) 2023-06-23T01:19:53.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:53 smithi134 conmon[143713]: 2023-06-23T01:19:53.055+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:22.090003+0000 front 2023-06-23T01:19:22.090048+0000 (oldest deadline 2023-06-23T01:19:47.390149+0000) 2023-06-23T01:19:54.126 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:53 smithi157 conmon[138256]: 2023-06-23T01:19:53.845+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:54.127 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:53 smithi157 conmon[138256]: 2023-06-23T01:19:53.845+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.766636+0000 front 2023-06-23T01:19:01.766694+0000 (oldest deadline 2023-06-23T01:19:24.666623+0000) 2023-06-23T01:19:54.127 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:53 smithi157 conmon[138256]: 2023-06-23T01:19:53.845+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.167466+0000 front 2023-06-23T01:19:05.167527+0000 (oldest deadline 2023-06-23T01:19:30.467672+0000) 2023-06-23T01:19:54.128 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:53 smithi157 conmon[138256]: 2023-06-23T01:19:53.845+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.770277+0000 front 2023-06-23T01:19:16.770175+0000 (oldest deadline 2023-06-23T01:19:40.270137+0000) 2023-06-23T01:19:54.128 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:53 smithi157 conmon[138256]: 2023-06-23T01:19:53.845+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:22.571301+0000 front 2023-06-23T01:19:22.571233+0000 (oldest deadline 2023-06-23T01:19:48.471321+0000) 2023-06-23T01:19:54.129 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:53 smithi157 conmon[145357]: 2023-06-23T01:19:53.682+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:54.129 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:53 smithi157 conmon[145357]: 2023-06-23T01:19:53.682+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:19:54.129 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:53 smithi157 conmon[145357]: 2023-06-23T01:19:53.682+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:19:54.129 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:53 smithi157 conmon[145357]: 2023-06-23T01:19:53.682+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.749536+0000 front 2023-06-23T01:19:25.749472+0000 (oldest deadline 2023-06-23T01:19:49.849573+0000) 2023-06-23T01:19:54.260 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:53 smithi120 conmon[117204]: 2023-06-23T01:19:53.844+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:54.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:53 smithi120 conmon[117204]: 2023-06-23T01:19:53.844+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:19:54.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:53 smithi120 conmon[117204]: 2023-06-23T01:19:53.844+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.597925+0000 front 2023-06-23T01:19:05.597907+0000 (oldest deadline 2023-06-23T01:19:31.497672+0000) 2023-06-23T01:19:54.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:53 smithi120 conmon[117204]: 2023-06-23T01:19:53.844+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.699443+0000 front 2023-06-23T01:19:16.699635+0000 (oldest deadline 2023-06-23T01:19:40.199557+0000) 2023-06-23T01:19:54.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:53 smithi120 conmon[117204]: 2023-06-23T01:19:53.844+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.500686+0000 front 2023-06-23T01:19:25.500560+0000 (oldest deadline 2023-06-23T01:19:46.600427+0000) 2023-06-23T01:19:54.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:54 smithi134 conmon[143713]: 2023-06-23T01:19:54.063+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:54.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:54 smithi134 conmon[143713]: 2023-06-23T01:19:54.063+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:19:54.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:54 smithi134 conmon[143713]: 2023-06-23T01:19:54.063+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:19:54.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:54 smithi134 conmon[143713]: 2023-06-23T01:19:54.063+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:17.389622+0000 front 2023-06-23T01:19:17.389746+0000 (oldest deadline 2023-06-23T01:19:42.089671+0000) 2023-06-23T01:19:54.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:54 smithi134 conmon[143713]: 2023-06-23T01:19:54.063+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:22.090003+0000 front 2023-06-23T01:19:22.090048+0000 (oldest deadline 2023-06-23T01:19:47.390149+0000) 2023-06-23T01:19:55.126 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:54 smithi157 conmon[138256]: 2023-06-23T01:19:54.856+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:55.127 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:54 smithi157 conmon[138256]: 2023-06-23T01:19:54.856+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.766636+0000 front 2023-06-23T01:19:01.766694+0000 (oldest deadline 2023-06-23T01:19:24.666623+0000) 2023-06-23T01:19:55.127 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:54 smithi157 conmon[138256]: 2023-06-23T01:19:54.856+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.167466+0000 front 2023-06-23T01:19:05.167527+0000 (oldest deadline 2023-06-23T01:19:30.467672+0000) 2023-06-23T01:19:55.127 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:54 smithi157 conmon[138256]: 2023-06-23T01:19:54.856+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.770277+0000 front 2023-06-23T01:19:16.770175+0000 (oldest deadline 2023-06-23T01:19:40.270137+0000) 2023-06-23T01:19:55.127 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:54 smithi157 conmon[138256]: 2023-06-23T01:19:54.856+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:22.571301+0000 front 2023-06-23T01:19:22.571233+0000 (oldest deadline 2023-06-23T01:19:48.471321+0000) 2023-06-23T01:19:55.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:54 smithi157 conmon[145357]: 2023-06-23T01:19:54.638+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:55.128 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:54 smithi157 conmon[145357]: 2023-06-23T01:19:54.638+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:19:55.128 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:54 smithi157 conmon[145357]: 2023-06-23T01:19:54.638+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:19:55.128 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:54 smithi157 conmon[145357]: 2023-06-23T01:19:54.638+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.749536+0000 front 2023-06-23T01:19:25.749472+0000 (oldest deadline 2023-06-23T01:19:49.849573+0000) 2023-06-23T01:19:55.260 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:54 smithi120 conmon[117204]: 2023-06-23T01:19:54.872+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:55.260 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:54 smithi120 conmon[117204]: 2023-06-23T01:19:54.872+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:19:55.260 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:54 smithi120 conmon[117204]: 2023-06-23T01:19:54.872+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.597925+0000 front 2023-06-23T01:19:05.597907+0000 (oldest deadline 2023-06-23T01:19:31.497672+0000) 2023-06-23T01:19:55.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:54 smithi120 conmon[117204]: 2023-06-23T01:19:54.872+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.699443+0000 front 2023-06-23T01:19:16.699635+0000 (oldest deadline 2023-06-23T01:19:40.199557+0000) 2023-06-23T01:19:55.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:54 smithi120 conmon[117204]: 2023-06-23T01:19:54.872+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.500686+0000 front 2023-06-23T01:19:25.500560+0000 (oldest deadline 2023-06-23T01:19:46.600427+0000) 2023-06-23T01:19:55.443 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:55 smithi134 conmon[143713]: 2023-06-23T01:19:55.060+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:55.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:55 smithi134 conmon[143713]: 2023-06-23T01:19:55.060+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:19:55.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:55 smithi134 conmon[143713]: 2023-06-23T01:19:55.060+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:19:55.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:55 smithi134 conmon[143713]: 2023-06-23T01:19:55.060+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:17.389622+0000 front 2023-06-23T01:19:17.389746+0000 (oldest deadline 2023-06-23T01:19:42.089671+0000) 2023-06-23T01:19:55.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:55 smithi134 conmon[143713]: 2023-06-23T01:19:55.060+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:22.090003+0000 front 2023-06-23T01:19:22.090048+0000 (oldest deadline 2023-06-23T01:19:47.390149+0000) 2023-06-23T01:19:55.901 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:55 smithi157 conmon[145357]: 2023-06-23T01:19:55.633+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:55.901 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:55 smithi157 conmon[145357]: 2023-06-23T01:19:55.633+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:19:55.901 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:55 smithi157 conmon[145357]: 2023-06-23T01:19:55.633+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:19:55.901 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:55 smithi157 conmon[145357]: 2023-06-23T01:19:55.633+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.749536+0000 front 2023-06-23T01:19:25.749472+0000 (oldest deadline 2023-06-23T01:19:49.849573+0000) 2023-06-23T01:19:55.901 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:55 smithi157 conmon[145357]: 2023-06-23T01:19:55.633+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:29.849868+0000 front 2023-06-23T01:19:29.849894+0000 (oldest deadline 2023-06-23T01:19:55.150061+0000) 2023-06-23T01:19:56.260 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:55 smithi120 conmon[117204]: 2023-06-23T01:19:55.869+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:56.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:55 smithi120 conmon[117204]: 2023-06-23T01:19:55.869+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:19:56.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:55 smithi120 conmon[117204]: 2023-06-23T01:19:55.869+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.597925+0000 front 2023-06-23T01:19:05.597907+0000 (oldest deadline 2023-06-23T01:19:31.497672+0000) 2023-06-23T01:19:56.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:55 smithi120 conmon[117204]: 2023-06-23T01:19:55.869+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.699443+0000 front 2023-06-23T01:19:16.699635+0000 (oldest deadline 2023-06-23T01:19:40.199557+0000) 2023-06-23T01:19:56.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:55 smithi120 conmon[117204]: 2023-06-23T01:19:55.869+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.500686+0000 front 2023-06-23T01:19:25.500560+0000 (oldest deadline 2023-06-23T01:19:46.600427+0000) 2023-06-23T01:19:56.376 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:55 smithi157 conmon[138256]: 2023-06-23T01:19:55.898+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:56.377 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:55 smithi157 conmon[138256]: 2023-06-23T01:19:55.898+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.766636+0000 front 2023-06-23T01:19:01.766694+0000 (oldest deadline 2023-06-23T01:19:24.666623+0000) 2023-06-23T01:19:56.377 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:55 smithi157 conmon[138256]: 2023-06-23T01:19:55.898+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.167466+0000 front 2023-06-23T01:19:05.167527+0000 (oldest deadline 2023-06-23T01:19:30.467672+0000) 2023-06-23T01:19:56.377 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:55 smithi157 conmon[138256]: 2023-06-23T01:19:55.898+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.770277+0000 front 2023-06-23T01:19:16.770175+0000 (oldest deadline 2023-06-23T01:19:40.270137+0000) 2023-06-23T01:19:56.377 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:55 smithi157 conmon[138256]: 2023-06-23T01:19:55.898+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:22.571301+0000 front 2023-06-23T01:19:22.571233+0000 (oldest deadline 2023-06-23T01:19:48.471321+0000) 2023-06-23T01:19:56.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:56 smithi134 conmon[143713]: 2023-06-23T01:19:56.057+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:56.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:56 smithi134 conmon[143713]: 2023-06-23T01:19:56.057+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:19:56.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:56 smithi134 conmon[143713]: 2023-06-23T01:19:56.057+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:19:56.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:56 smithi134 conmon[143713]: 2023-06-23T01:19:56.057+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:17.389622+0000 front 2023-06-23T01:19:17.389746+0000 (oldest deadline 2023-06-23T01:19:42.089671+0000) 2023-06-23T01:19:56.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:56 smithi134 conmon[143713]: 2023-06-23T01:19:56.057+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:22.090003+0000 front 2023-06-23T01:19:22.090048+0000 (oldest deadline 2023-06-23T01:19:47.390149+0000) 2023-06-23T01:19:56.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:56 smithi134 conmon[143713]: 2023-06-23T01:19:56.057+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:31.391457+0000 front 2023-06-23T01:19:31.391562+0000 (oldest deadline 2023-06-23T01:19:55.391959+0000) 2023-06-23T01:19:57.126 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:56 smithi157 conmon[145357]: 2023-06-23T01:19:56.642+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:57.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:56 smithi157 conmon[145357]: 2023-06-23T01:19:56.642+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:19:57.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:56 smithi157 conmon[145357]: 2023-06-23T01:19:56.642+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:19:57.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:56 smithi157 conmon[145357]: 2023-06-23T01:19:56.642+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.749536+0000 front 2023-06-23T01:19:25.749472+0000 (oldest deadline 2023-06-23T01:19:49.849573+0000) 2023-06-23T01:19:57.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:56 smithi157 conmon[145357]: 2023-06-23T01:19:56.642+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:29.849868+0000 front 2023-06-23T01:19:29.849894+0000 (oldest deadline 2023-06-23T01:19:55.150061+0000) 2023-06-23T01:19:57.128 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:56 smithi157 conmon[138256]: 2023-06-23T01:19:56.886+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:57.128 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:56 smithi157 conmon[138256]: 2023-06-23T01:19:56.886+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.766636+0000 front 2023-06-23T01:19:01.766694+0000 (oldest deadline 2023-06-23T01:19:24.666623+0000) 2023-06-23T01:19:57.128 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:56 smithi157 conmon[138256]: 2023-06-23T01:19:56.886+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.167466+0000 front 2023-06-23T01:19:05.167527+0000 (oldest deadline 2023-06-23T01:19:30.467672+0000) 2023-06-23T01:19:57.128 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:56 smithi157 conmon[138256]: 2023-06-23T01:19:56.886+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.770277+0000 front 2023-06-23T01:19:16.770175+0000 (oldest deadline 2023-06-23T01:19:40.270137+0000) 2023-06-23T01:19:57.128 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:56 smithi157 conmon[138256]: 2023-06-23T01:19:56.886+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:22.571301+0000 front 2023-06-23T01:19:22.571233+0000 (oldest deadline 2023-06-23T01:19:48.471321+0000) 2023-06-23T01:19:57.128 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:56 smithi157 conmon[138256]: 2023-06-23T01:19:56.886+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:32.572465+0000 front 2023-06-23T01:19:32.572489+0000 (oldest deadline 2023-06-23T01:19:56.072742+0000) 2023-06-23T01:19:57.260 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:56 smithi120 conmon[117204]: 2023-06-23T01:19:56.847+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:57.260 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:56 smithi120 conmon[117204]: 2023-06-23T01:19:56.847+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:19:57.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:56 smithi120 conmon[117204]: 2023-06-23T01:19:56.847+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.597925+0000 front 2023-06-23T01:19:05.597907+0000 (oldest deadline 2023-06-23T01:19:31.497672+0000) 2023-06-23T01:19:57.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:56 smithi120 conmon[117204]: 2023-06-23T01:19:56.847+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.699443+0000 front 2023-06-23T01:19:16.699635+0000 (oldest deadline 2023-06-23T01:19:40.199557+0000) 2023-06-23T01:19:57.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:56 smithi120 conmon[117204]: 2023-06-23T01:19:56.847+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.500686+0000 front 2023-06-23T01:19:25.500560+0000 (oldest deadline 2023-06-23T01:19:46.600427+0000) 2023-06-23T01:19:57.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:57 smithi134 conmon[143713]: 2023-06-23T01:19:57.079+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:57.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:57 smithi134 conmon[143713]: 2023-06-23T01:19:57.079+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:19:57.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:57 smithi134 conmon[143713]: 2023-06-23T01:19:57.079+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:19:57.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:57 smithi134 conmon[143713]: 2023-06-23T01:19:57.079+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:17.389622+0000 front 2023-06-23T01:19:17.389746+0000 (oldest deadline 2023-06-23T01:19:42.089671+0000) 2023-06-23T01:19:57.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:57 smithi134 conmon[143713]: 2023-06-23T01:19:57.079+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:22.090003+0000 front 2023-06-23T01:19:22.090048+0000 (oldest deadline 2023-06-23T01:19:47.390149+0000) 2023-06-23T01:19:57.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:57 smithi134 conmon[143713]: 2023-06-23T01:19:57.079+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:31.391457+0000 front 2023-06-23T01:19:31.391562+0000 (oldest deadline 2023-06-23T01:19:55.391959+0000) 2023-06-23T01:19:58.126 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:57 smithi157 conmon[138256]: 2023-06-23T01:19:57.869+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.265111+0000 front 2023-06-23T01:18:51.264909+0000 (oldest deadline 2023-06-23T01:19:17.164877+0000) 2023-06-23T01:19:58.127 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:57 smithi157 conmon[138256]: 2023-06-23T01:19:57.869+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:01.766636+0000 front 2023-06-23T01:19:01.766694+0000 (oldest deadline 2023-06-23T01:19:24.666623+0000) 2023-06-23T01:19:58.127 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:57 smithi157 conmon[138256]: 2023-06-23T01:19:57.869+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.167466+0000 front 2023-06-23T01:19:05.167527+0000 (oldest deadline 2023-06-23T01:19:30.467672+0000) 2023-06-23T01:19:58.127 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:57 smithi157 conmon[138256]: 2023-06-23T01:19:57.869+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.770277+0000 front 2023-06-23T01:19:16.770175+0000 (oldest deadline 2023-06-23T01:19:40.270137+0000) 2023-06-23T01:19:58.127 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:57 smithi157 conmon[138256]: 2023-06-23T01:19:57.869+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:22.571301+0000 front 2023-06-23T01:19:22.571233+0000 (oldest deadline 2023-06-23T01:19:48.471321+0000) 2023-06-23T01:19:58.127 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:57 smithi157 conmon[138256]: 2023-06-23T01:19:57.869+0000 7fd2e6449700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:32.572465+0000 front 2023-06-23T01:19:32.572489+0000 (oldest deadline 2023-06-23T01:19:56.072742+0000) 2023-06-23T01:19:58.128 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:57 smithi157 conmon[145357]: 2023-06-23T01:19:57.636+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:58.128 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:57 smithi157 conmon[145357]: 2023-06-23T01:19:57.636+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:19:58.128 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:57 smithi157 conmon[145357]: 2023-06-23T01:19:57.636+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:19:58.128 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:57 smithi157 conmon[145357]: 2023-06-23T01:19:57.636+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.749536+0000 front 2023-06-23T01:19:25.749472+0000 (oldest deadline 2023-06-23T01:19:49.849573+0000) 2023-06-23T01:19:58.128 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:57 smithi157 conmon[145357]: 2023-06-23T01:19:57.636+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:29.849868+0000 front 2023-06-23T01:19:29.849894+0000 (oldest deadline 2023-06-23T01:19:55.150061+0000) 2023-06-23T01:19:58.260 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:57 smithi120 conmon[117204]: 2023-06-23T01:19:57.859+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:58.260 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:57 smithi120 conmon[117204]: 2023-06-23T01:19:57.859+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:19:58.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:57 smithi120 conmon[117204]: 2023-06-23T01:19:57.859+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.597925+0000 front 2023-06-23T01:19:05.597907+0000 (oldest deadline 2023-06-23T01:19:31.497672+0000) 2023-06-23T01:19:58.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:57 smithi120 conmon[117204]: 2023-06-23T01:19:57.859+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.699443+0000 front 2023-06-23T01:19:16.699635+0000 (oldest deadline 2023-06-23T01:19:40.199557+0000) 2023-06-23T01:19:58.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:57 smithi120 conmon[117204]: 2023-06-23T01:19:57.859+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.500686+0000 front 2023-06-23T01:19:25.500560+0000 (oldest deadline 2023-06-23T01:19:46.600427+0000) 2023-06-23T01:19:58.443 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:58 smithi134 conmon[143713]: 2023-06-23T01:19:58.127+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:58.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:58 smithi134 conmon[143713]: 2023-06-23T01:19:58.127+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:19:58.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:58 smithi134 conmon[143713]: 2023-06-23T01:19:58.127+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:19:58.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:58 smithi134 conmon[143713]: 2023-06-23T01:19:58.127+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:17.389622+0000 front 2023-06-23T01:19:17.389746+0000 (oldest deadline 2023-06-23T01:19:42.089671+0000) 2023-06-23T01:19:58.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:58 smithi134 conmon[143713]: 2023-06-23T01:19:58.127+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:22.090003+0000 front 2023-06-23T01:19:22.090048+0000 (oldest deadline 2023-06-23T01:19:47.390149+0000) 2023-06-23T01:19:58.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:58 smithi134 conmon[143713]: 2023-06-23T01:19:58.127+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:31.391457+0000 front 2023-06-23T01:19:31.391562+0000 (oldest deadline 2023-06-23T01:19:55.391959+0000) 2023-06-23T01:19:58.876 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:58 smithi157 conmon[145357]: 2023-06-23T01:19:58.620+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:58.876 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:58 smithi157 conmon[145357]: 2023-06-23T01:19:58.620+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:19:58.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:58 smithi157 conmon[145357]: 2023-06-23T01:19:58.620+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:19:58.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:58 smithi157 conmon[145357]: 2023-06-23T01:19:58.620+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.749536+0000 front 2023-06-23T01:19:25.749472+0000 (oldest deadline 2023-06-23T01:19:49.849573+0000) 2023-06-23T01:19:58.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:58 smithi157 conmon[145357]: 2023-06-23T01:19:58.620+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:29.849868+0000 front 2023-06-23T01:19:29.849894+0000 (oldest deadline 2023-06-23T01:19:55.150061+0000) 2023-06-23T01:19:59.260 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:58 smithi120 conmon[117204]: 2023-06-23T01:19:58.889+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:19:59.260 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:58 smithi120 conmon[117204]: 2023-06-23T01:19:58.889+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:19:59.260 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:58 smithi120 conmon[117204]: 2023-06-23T01:19:58.889+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.597925+0000 front 2023-06-23T01:19:05.597907+0000 (oldest deadline 2023-06-23T01:19:31.497672+0000) 2023-06-23T01:19:59.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:58 smithi120 conmon[117204]: 2023-06-23T01:19:58.889+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.699443+0000 front 2023-06-23T01:19:16.699635+0000 (oldest deadline 2023-06-23T01:19:40.199557+0000) 2023-06-23T01:19:59.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:58 smithi120 conmon[117204]: 2023-06-23T01:19:58.889+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.500686+0000 front 2023-06-23T01:19:25.500560+0000 (oldest deadline 2023-06-23T01:19:46.600427+0000) 2023-06-23T01:19:59.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:58 smithi120 conmon[117204]: 2023-06-23T01:19:58.889+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:32.402536+0000 front 2023-06-23T01:19:32.402630+0000 (oldest deadline 2023-06-23T01:19:58.302556+0000) 2023-06-23T01:19:59.376 INFO:journalctl@ceph.osd.8.smithi157.stdout:Jun 23 01:19:58 smithi157 bash[154989]: ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c-osd-8 2023-06-23T01:19:59.443 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:59 smithi134 conmon[143713]: 2023-06-23T01:19:59.127+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:19:59.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:59 smithi134 conmon[143713]: 2023-06-23T01:19:59.127+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:19:59.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:59 smithi134 conmon[143713]: 2023-06-23T01:19:59.127+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:19:59.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:59 smithi134 conmon[143713]: 2023-06-23T01:19:59.127+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:17.389622+0000 front 2023-06-23T01:19:17.389746+0000 (oldest deadline 2023-06-23T01:19:42.089671+0000) 2023-06-23T01:19:59.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:59 smithi134 conmon[143713]: 2023-06-23T01:19:59.127+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:22.090003+0000 front 2023-06-23T01:19:22.090048+0000 (oldest deadline 2023-06-23T01:19:47.390149+0000) 2023-06-23T01:19:59.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:19:59 smithi134 conmon[143713]: 2023-06-23T01:19:59.127+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:31.391457+0000 front 2023-06-23T01:19:31.391562+0000 (oldest deadline 2023-06-23T01:19:55.391959+0000) 2023-06-23T01:19:59.876 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:59 smithi157 conmon[145357]: 2023-06-23T01:19:59.571+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:19:59.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:59 smithi157 conmon[145357]: 2023-06-23T01:19:59.571+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:19:59.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:59 smithi157 conmon[145357]: 2023-06-23T01:19:59.571+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:19:59.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:59 smithi157 conmon[145357]: 2023-06-23T01:19:59.571+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.749536+0000 front 2023-06-23T01:19:25.749472+0000 (oldest deadline 2023-06-23T01:19:49.849573+0000) 2023-06-23T01:19:59.878 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:19:59 smithi157 conmon[145357]: 2023-06-23T01:19:59.571+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:29.849868+0000 front 2023-06-23T01:19:29.849894+0000 (oldest deadline 2023-06-23T01:19:55.150061+0000) 2023-06-23T01:20:00.260 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:59 smithi120 conmon[117204]: 2023-06-23T01:19:59.914+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:20:00.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:59 smithi120 conmon[117204]: 2023-06-23T01:19:59.914+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:20:00.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:59 smithi120 conmon[117204]: 2023-06-23T01:19:59.914+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.597925+0000 front 2023-06-23T01:19:05.597907+0000 (oldest deadline 2023-06-23T01:19:31.497672+0000) 2023-06-23T01:20:00.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:59 smithi120 conmon[117204]: 2023-06-23T01:19:59.914+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.699443+0000 front 2023-06-23T01:19:16.699635+0000 (oldest deadline 2023-06-23T01:19:40.199557+0000) 2023-06-23T01:20:00.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:59 smithi120 conmon[117204]: 2023-06-23T01:19:59.914+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.500686+0000 front 2023-06-23T01:19:25.500560+0000 (oldest deadline 2023-06-23T01:19:46.600427+0000) 2023-06-23T01:20:00.262 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:19:59 smithi120 conmon[117204]: 2023-06-23T01:19:59.914+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:32.402536+0000 front 2023-06-23T01:19:32.402630+0000 (oldest deadline 2023-06-23T01:19:58.302556+0000) 2023-06-23T01:20:00.443 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:00 smithi134 conmon[143713]: 2023-06-23T01:20:00.159+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:20:00.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:00 smithi134 conmon[143713]: 2023-06-23T01:20:00.159+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:20:00.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:00 smithi134 conmon[143713]: 2023-06-23T01:20:00.159+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:20:00.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:00 smithi134 conmon[143713]: 2023-06-23T01:20:00.159+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:17.389622+0000 front 2023-06-23T01:19:17.389746+0000 (oldest deadline 2023-06-23T01:19:42.089671+0000) 2023-06-23T01:20:00.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:00 smithi134 conmon[143713]: 2023-06-23T01:20:00.159+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:22.090003+0000 front 2023-06-23T01:19:22.090048+0000 (oldest deadline 2023-06-23T01:19:47.390149+0000) 2023-06-23T01:20:00.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:00 smithi134 conmon[143713]: 2023-06-23T01:20:00.159+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:31.391457+0000 front 2023-06-23T01:19:31.391562+0000 (oldest deadline 2023-06-23T01:19:55.391959+0000) 2023-06-23T01:20:00.646 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@osd.8.service' 2023-06-23T01:20:00.685 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T01:20:00.686 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-06-23T01:20:00.686 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-06-23T01:20:00.687 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@osd.9 2023-06-23T01:20:00.876 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:00 smithi157 conmon[145357]: 2023-06-23T01:20:00.612+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:20:00.876 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:00 smithi157 conmon[145357]: 2023-06-23T01:20:00.612+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:20:00.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:00 smithi157 conmon[145357]: 2023-06-23T01:20:00.612+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:20:00.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:00 smithi157 conmon[145357]: 2023-06-23T01:20:00.612+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.749536+0000 front 2023-06-23T01:19:25.749472+0000 (oldest deadline 2023-06-23T01:19:49.849573+0000) 2023-06-23T01:20:00.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:00 smithi157 conmon[145357]: 2023-06-23T01:20:00.612+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:29.849868+0000 front 2023-06-23T01:19:29.849894+0000 (oldest deadline 2023-06-23T01:19:55.150061+0000) 2023-06-23T01:20:01.010 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:00 smithi120 systemd[1]: Stopping Ceph osd.9 for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:20:01.011 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:00 smithi120 conmon[117204]: 2023-06-23T01:20:00.922+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:20:01.011 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:00 smithi120 conmon[117204]: 2023-06-23T01:20:00.922+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:20:01.011 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:00 smithi120 conmon[117204]: 2023-06-23T01:20:00.922+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.597925+0000 front 2023-06-23T01:19:05.597907+0000 (oldest deadline 2023-06-23T01:19:31.497672+0000) 2023-06-23T01:20:01.011 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:00 smithi120 conmon[117204]: 2023-06-23T01:20:00.922+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.699443+0000 front 2023-06-23T01:19:16.699635+0000 (oldest deadline 2023-06-23T01:19:40.199557+0000) 2023-06-23T01:20:01.011 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:00 smithi120 conmon[117204]: 2023-06-23T01:20:00.922+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.500686+0000 front 2023-06-23T01:19:25.500560+0000 (oldest deadline 2023-06-23T01:19:46.600427+0000) 2023-06-23T01:20:01.011 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:00 smithi120 conmon[117204]: 2023-06-23T01:20:00.922+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:32.402536+0000 front 2023-06-23T01:19:32.402630+0000 (oldest deadline 2023-06-23T01:19:58.302556+0000) 2023-06-23T01:20:01.443 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:01 smithi134 conmon[143713]: 2023-06-23T01:20:01.152+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:20:01.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:01 smithi134 conmon[143713]: 2023-06-23T01:20:01.152+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:20:01.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:01 smithi134 conmon[143713]: 2023-06-23T01:20:01.152+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:20:01.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:01 smithi134 conmon[143713]: 2023-06-23T01:20:01.152+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:17.389622+0000 front 2023-06-23T01:19:17.389746+0000 (oldest deadline 2023-06-23T01:19:42.089671+0000) 2023-06-23T01:20:01.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:01 smithi134 conmon[143713]: 2023-06-23T01:20:01.152+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:22.090003+0000 front 2023-06-23T01:19:22.090048+0000 (oldest deadline 2023-06-23T01:19:47.390149+0000) 2023-06-23T01:20:01.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:01 smithi134 conmon[143713]: 2023-06-23T01:20:01.152+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:31.391457+0000 front 2023-06-23T01:19:31.391562+0000 (oldest deadline 2023-06-23T01:19:55.391959+0000) 2023-06-23T01:20:01.510 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:01 smithi120 conmon[117204]: 2023-06-23T01:20:01.192+0000 7f748b8ba700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T01:20:01.510 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:01 smithi120 conmon[117204]: 2023-06-23T01:20:01.192+0000 7f748b8ba700 -1 osd.9 85 *** Got signal Terminated *** 2023-06-23T01:20:01.510 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:01 smithi120 conmon[117204]: 2023-06-23T01:20:01.192+0000 7f748b8ba700 -1 osd.9 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-23T01:20:01.876 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:01 smithi157 conmon[145357]: 2023-06-23T01:20:01.617+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:20:01.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:01 smithi157 conmon[145357]: 2023-06-23T01:20:01.617+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:20:01.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:01 smithi157 conmon[145357]: 2023-06-23T01:20:01.617+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:20:01.878 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:01 smithi157 conmon[145357]: 2023-06-23T01:20:01.617+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.749536+0000 front 2023-06-23T01:19:25.749472+0000 (oldest deadline 2023-06-23T01:19:49.849573+0000) 2023-06-23T01:20:01.878 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:01 smithi157 conmon[145357]: 2023-06-23T01:20:01.617+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:29.849868+0000 front 2023-06-23T01:19:29.849894+0000 (oldest deadline 2023-06-23T01:19:55.150061+0000) 2023-06-23T01:20:02.260 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:01 smithi120 conmon[117204]: 2023-06-23T01:20:01.906+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:20:02.260 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:01 smithi120 conmon[117204]: 2023-06-23T01:20:01.906+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:20:02.260 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:01 smithi120 conmon[117204]: 2023-06-23T01:20:01.906+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.597925+0000 front 2023-06-23T01:19:05.597907+0000 (oldest deadline 2023-06-23T01:19:31.497672+0000) 2023-06-23T01:20:02.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:01 smithi120 conmon[117204]: 2023-06-23T01:20:01.906+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.699443+0000 front 2023-06-23T01:19:16.699635+0000 (oldest deadline 2023-06-23T01:19:40.199557+0000) 2023-06-23T01:20:02.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:01 smithi120 conmon[117204]: 2023-06-23T01:20:01.906+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.500686+0000 front 2023-06-23T01:19:25.500560+0000 (oldest deadline 2023-06-23T01:19:46.600427+0000) 2023-06-23T01:20:02.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:01 smithi120 conmon[117204]: 2023-06-23T01:20:01.906+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:32.402536+0000 front 2023-06-23T01:19:32.402630+0000 (oldest deadline 2023-06-23T01:19:58.302556+0000) 2023-06-23T01:20:02.443 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:02 smithi134 conmon[143713]: 2023-06-23T01:20:02.141+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:20:02.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:02 smithi134 conmon[143713]: 2023-06-23T01:20:02.141+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:20:02.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:02 smithi134 conmon[143713]: 2023-06-23T01:20:02.141+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:20:02.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:02 smithi134 conmon[143713]: 2023-06-23T01:20:02.141+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:17.389622+0000 front 2023-06-23T01:19:17.389746+0000 (oldest deadline 2023-06-23T01:19:42.089671+0000) 2023-06-23T01:20:02.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:02 smithi134 conmon[143713]: 2023-06-23T01:20:02.141+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:22.090003+0000 front 2023-06-23T01:19:22.090048+0000 (oldest deadline 2023-06-23T01:19:47.390149+0000) 2023-06-23T01:20:02.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:02 smithi134 conmon[143713]: 2023-06-23T01:20:02.141+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:31.391457+0000 front 2023-06-23T01:19:31.391562+0000 (oldest deadline 2023-06-23T01:19:55.391959+0000) 2023-06-23T01:20:02.876 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:02 smithi157 conmon[145357]: 2023-06-23T01:20:02.602+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:20:02.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:02 smithi157 conmon[145357]: 2023-06-23T01:20:02.602+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:20:02.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:02 smithi157 conmon[145357]: 2023-06-23T01:20:02.602+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:20:02.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:02 smithi157 conmon[145357]: 2023-06-23T01:20:02.602+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.749536+0000 front 2023-06-23T01:19:25.749472+0000 (oldest deadline 2023-06-23T01:19:49.849573+0000) 2023-06-23T01:20:02.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:02 smithi157 conmon[145357]: 2023-06-23T01:20:02.602+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:29.849868+0000 front 2023-06-23T01:19:29.849894+0000 (oldest deadline 2023-06-23T01:19:55.150061+0000) 2023-06-23T01:20:03.260 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:02 smithi120 conmon[117204]: 2023-06-23T01:20:02.887+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:20:03.260 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:02 smithi120 conmon[117204]: 2023-06-23T01:20:02.887+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:20:03.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:02 smithi120 conmon[117204]: 2023-06-23T01:20:02.887+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.597925+0000 front 2023-06-23T01:19:05.597907+0000 (oldest deadline 2023-06-23T01:19:31.497672+0000) 2023-06-23T01:20:03.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:02 smithi120 conmon[117204]: 2023-06-23T01:20:02.887+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.699443+0000 front 2023-06-23T01:19:16.699635+0000 (oldest deadline 2023-06-23T01:19:40.199557+0000) 2023-06-23T01:20:03.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:02 smithi120 conmon[117204]: 2023-06-23T01:20:02.887+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.500686+0000 front 2023-06-23T01:19:25.500560+0000 (oldest deadline 2023-06-23T01:19:46.600427+0000) 2023-06-23T01:20:03.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:02 smithi120 conmon[117204]: 2023-06-23T01:20:02.887+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:32.402536+0000 front 2023-06-23T01:19:32.402630+0000 (oldest deadline 2023-06-23T01:19:58.302556+0000) 2023-06-23T01:20:03.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:03 smithi134 conmon[143713]: 2023-06-23T01:20:03.140+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:20:03.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:03 smithi134 conmon[143713]: 2023-06-23T01:20:03.140+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:20:03.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:03 smithi134 conmon[143713]: 2023-06-23T01:20:03.140+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:20:03.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:03 smithi134 conmon[143713]: 2023-06-23T01:20:03.140+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:17.389622+0000 front 2023-06-23T01:19:17.389746+0000 (oldest deadline 2023-06-23T01:19:42.089671+0000) 2023-06-23T01:20:03.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:03 smithi134 conmon[143713]: 2023-06-23T01:20:03.140+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:22.090003+0000 front 2023-06-23T01:19:22.090048+0000 (oldest deadline 2023-06-23T01:19:47.390149+0000) 2023-06-23T01:20:03.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:03 smithi134 conmon[143713]: 2023-06-23T01:20:03.140+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:31.391457+0000 front 2023-06-23T01:19:31.391562+0000 (oldest deadline 2023-06-23T01:19:55.391959+0000) 2023-06-23T01:20:03.876 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:03 smithi157 conmon[145357]: 2023-06-23T01:20:03.593+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:20:03.876 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:03 smithi157 conmon[145357]: 2023-06-23T01:20:03.593+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:20:03.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:03 smithi157 conmon[145357]: 2023-06-23T01:20:03.593+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:20:03.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:03 smithi157 conmon[145357]: 2023-06-23T01:20:03.593+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.749536+0000 front 2023-06-23T01:19:25.749472+0000 (oldest deadline 2023-06-23T01:19:49.849573+0000) 2023-06-23T01:20:03.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:03 smithi157 conmon[145357]: 2023-06-23T01:20:03.593+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:29.849868+0000 front 2023-06-23T01:19:29.849894+0000 (oldest deadline 2023-06-23T01:19:55.150061+0000) 2023-06-23T01:20:03.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:03 smithi157 conmon[145357]: 2023-06-23T01:20:03.593+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2023-06-23T01:19:36.850984+0000 front 2023-06-23T01:19:36.850994+0000 (oldest deadline 2023-06-23T01:20:02.751055+0000) 2023-06-23T01:20:04.260 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:03 smithi120 conmon[117204]: 2023-06-23T01:20:03.916+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:20:04.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:03 smithi120 conmon[117204]: 2023-06-23T01:20:03.916+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:20:04.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:03 smithi120 conmon[117204]: 2023-06-23T01:20:03.916+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.597925+0000 front 2023-06-23T01:19:05.597907+0000 (oldest deadline 2023-06-23T01:19:31.497672+0000) 2023-06-23T01:20:04.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:03 smithi120 conmon[117204]: 2023-06-23T01:20:03.916+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.699443+0000 front 2023-06-23T01:19:16.699635+0000 (oldest deadline 2023-06-23T01:19:40.199557+0000) 2023-06-23T01:20:04.262 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:03 smithi120 conmon[117204]: 2023-06-23T01:20:03.916+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.500686+0000 front 2023-06-23T01:19:25.500560+0000 (oldest deadline 2023-06-23T01:19:46.600427+0000) 2023-06-23T01:20:04.262 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:03 smithi120 conmon[117204]: 2023-06-23T01:20:03.916+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:32.402536+0000 front 2023-06-23T01:19:32.402630+0000 (oldest deadline 2023-06-23T01:19:58.302556+0000) 2023-06-23T01:20:04.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:04 smithi134 conmon[143713]: 2023-06-23T01:20:04.104+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:20:04.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:04 smithi134 conmon[143713]: 2023-06-23T01:20:04.104+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:20:04.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:04 smithi134 conmon[143713]: 2023-06-23T01:20:04.104+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:20:04.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:04 smithi134 conmon[143713]: 2023-06-23T01:20:04.104+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:17.389622+0000 front 2023-06-23T01:19:17.389746+0000 (oldest deadline 2023-06-23T01:19:42.089671+0000) 2023-06-23T01:20:04.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:04 smithi134 conmon[143713]: 2023-06-23T01:20:04.104+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:22.090003+0000 front 2023-06-23T01:19:22.090048+0000 (oldest deadline 2023-06-23T01:19:47.390149+0000) 2023-06-23T01:20:04.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:04 smithi134 conmon[143713]: 2023-06-23T01:20:04.104+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:31.391457+0000 front 2023-06-23T01:19:31.391562+0000 (oldest deadline 2023-06-23T01:19:55.391959+0000) 2023-06-23T01:20:04.876 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:04 smithi157 conmon[145357]: 2023-06-23T01:20:04.586+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:20:04.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:04 smithi157 conmon[145357]: 2023-06-23T01:20:04.586+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:20:04.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:04 smithi157 conmon[145357]: 2023-06-23T01:20:04.586+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:20:04.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:04 smithi157 conmon[145357]: 2023-06-23T01:20:04.586+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.749536+0000 front 2023-06-23T01:19:25.749472+0000 (oldest deadline 2023-06-23T01:19:49.849573+0000) 2023-06-23T01:20:04.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:04 smithi157 conmon[145357]: 2023-06-23T01:20:04.586+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:29.849868+0000 front 2023-06-23T01:19:29.849894+0000 (oldest deadline 2023-06-23T01:19:55.150061+0000) 2023-06-23T01:20:04.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:04 smithi157 conmon[145357]: 2023-06-23T01:20:04.586+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2023-06-23T01:19:36.850984+0000 front 2023-06-23T01:19:36.850994+0000 (oldest deadline 2023-06-23T01:20:02.751055+0000) 2023-06-23T01:20:05.260 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:04 smithi120 conmon[117204]: 2023-06-23T01:20:04.905+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:20:05.260 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:04 smithi120 conmon[117204]: 2023-06-23T01:20:04.905+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:20:05.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:04 smithi120 conmon[117204]: 2023-06-23T01:20:04.905+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.597925+0000 front 2023-06-23T01:19:05.597907+0000 (oldest deadline 2023-06-23T01:19:31.497672+0000) 2023-06-23T01:20:05.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:04 smithi120 conmon[117204]: 2023-06-23T01:20:04.905+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.699443+0000 front 2023-06-23T01:19:16.699635+0000 (oldest deadline 2023-06-23T01:19:40.199557+0000) 2023-06-23T01:20:05.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:04 smithi120 conmon[117204]: 2023-06-23T01:20:04.905+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.500686+0000 front 2023-06-23T01:19:25.500560+0000 (oldest deadline 2023-06-23T01:19:46.600427+0000) 2023-06-23T01:20:05.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:04 smithi120 conmon[117204]: 2023-06-23T01:20:04.905+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:32.402536+0000 front 2023-06-23T01:19:32.402630+0000 (oldest deadline 2023-06-23T01:19:58.302556+0000) 2023-06-23T01:20:05.443 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:05 smithi134 conmon[143713]: 2023-06-23T01:20:05.089+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:20:05.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:05 smithi134 conmon[143713]: 2023-06-23T01:20:05.089+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:20:05.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:05 smithi134 conmon[143713]: 2023-06-23T01:20:05.089+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:20:05.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:05 smithi134 conmon[143713]: 2023-06-23T01:20:05.089+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:17.389622+0000 front 2023-06-23T01:19:17.389746+0000 (oldest deadline 2023-06-23T01:19:42.089671+0000) 2023-06-23T01:20:05.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:05 smithi134 conmon[143713]: 2023-06-23T01:20:05.089+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:22.090003+0000 front 2023-06-23T01:19:22.090048+0000 (oldest deadline 2023-06-23T01:19:47.390149+0000) 2023-06-23T01:20:05.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:05 smithi134 conmon[143713]: 2023-06-23T01:20:05.089+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:31.391457+0000 front 2023-06-23T01:19:31.391562+0000 (oldest deadline 2023-06-23T01:19:55.391959+0000) 2023-06-23T01:20:06.126 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:05 smithi157 conmon[145357]: 2023-06-23T01:20:05.626+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:20:06.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:05 smithi157 conmon[145357]: 2023-06-23T01:20:05.626+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:20:06.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:05 smithi157 conmon[145357]: 2023-06-23T01:20:05.626+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:20:06.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:05 smithi157 conmon[145357]: 2023-06-23T01:20:05.626+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.749536+0000 front 2023-06-23T01:19:25.749472+0000 (oldest deadline 2023-06-23T01:19:49.849573+0000) 2023-06-23T01:20:06.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:05 smithi157 conmon[145357]: 2023-06-23T01:20:05.626+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:29.849868+0000 front 2023-06-23T01:19:29.849894+0000 (oldest deadline 2023-06-23T01:19:55.150061+0000) 2023-06-23T01:20:06.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:05 smithi157 conmon[145357]: 2023-06-23T01:20:05.626+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2023-06-23T01:19:36.850984+0000 front 2023-06-23T01:19:36.850994+0000 (oldest deadline 2023-06-23T01:20:02.751055+0000) 2023-06-23T01:20:06.260 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:05 smithi120 conmon[117204]: 2023-06-23T01:20:05.920+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:52.293695+0000 front 2023-06-23T01:18:52.293633+0000 (oldest deadline 2023-06-23T01:19:18.094773+0000) 2023-06-23T01:20:06.260 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:05 smithi120 conmon[117204]: 2023-06-23T01:20:05.920+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.897157+0000 front 2023-06-23T01:19:00.897126+0000 (oldest deadline 2023-06-23T01:19:25.597228+0000) 2023-06-23T01:20:06.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:05 smithi120 conmon[117204]: 2023-06-23T01:20:05.920+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:05.597925+0000 front 2023-06-23T01:19:05.597907+0000 (oldest deadline 2023-06-23T01:19:31.497672+0000) 2023-06-23T01:20:06.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:05 smithi120 conmon[117204]: 2023-06-23T01:20:05.920+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:16.699443+0000 front 2023-06-23T01:19:16.699635+0000 (oldest deadline 2023-06-23T01:19:40.199557+0000) 2023-06-23T01:20:06.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:05 smithi120 conmon[117204]: 2023-06-23T01:20:05.920+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.500686+0000 front 2023-06-23T01:19:25.500560+0000 (oldest deadline 2023-06-23T01:19:46.600427+0000) 2023-06-23T01:20:06.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:05 smithi120 conmon[117204]: 2023-06-23T01:20:05.920+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:32.402536+0000 front 2023-06-23T01:19:32.402630+0000 (oldest deadline 2023-06-23T01:19:58.302556+0000) 2023-06-23T01:20:06.261 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:05 smithi120 conmon[117204]: 2023-06-23T01:20:05.920+0000 7f7484eb2700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2023-06-23T01:19:40.003444+0000 front 2023-06-23T01:19:40.003492+0000 (oldest deadline 2023-06-23T01:20:05.903623+0000) 2023-06-23T01:20:06.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:06 smithi134 conmon[143713]: 2023-06-23T01:20:06.125+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:20:06.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:06 smithi134 conmon[143713]: 2023-06-23T01:20:06.125+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:20:06.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:06 smithi134 conmon[143713]: 2023-06-23T01:20:06.125+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:20:06.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:06 smithi134 conmon[143713]: 2023-06-23T01:20:06.125+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:17.389622+0000 front 2023-06-23T01:19:17.389746+0000 (oldest deadline 2023-06-23T01:19:42.089671+0000) 2023-06-23T01:20:06.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:06 smithi134 conmon[143713]: 2023-06-23T01:20:06.125+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:22.090003+0000 front 2023-06-23T01:19:22.090048+0000 (oldest deadline 2023-06-23T01:19:47.390149+0000) 2023-06-23T01:20:06.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:06 smithi134 conmon[143713]: 2023-06-23T01:20:06.125+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:31.391457+0000 front 2023-06-23T01:19:31.391562+0000 (oldest deadline 2023-06-23T01:19:55.391959+0000) 2023-06-23T01:20:06.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:06 smithi134 conmon[143713]: 2023-06-23T01:20:06.125+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2023-06-23T01:19:41.293079+0000 front 2023-06-23T01:19:41.293164+0000 (oldest deadline 2023-06-23T01:20:05.992903+0000) 2023-06-23T01:20:07.126 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:06 smithi157 conmon[145357]: 2023-06-23T01:20:06.674+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:20:07.126 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:06 smithi157 conmon[145357]: 2023-06-23T01:20:06.674+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:20:07.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:06 smithi157 conmon[145357]: 2023-06-23T01:20:06.674+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:20:07.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:06 smithi157 conmon[145357]: 2023-06-23T01:20:06.674+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.749536+0000 front 2023-06-23T01:19:25.749472+0000 (oldest deadline 2023-06-23T01:19:49.849573+0000) 2023-06-23T01:20:07.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:06 smithi157 conmon[145357]: 2023-06-23T01:20:06.674+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:29.849868+0000 front 2023-06-23T01:19:29.849894+0000 (oldest deadline 2023-06-23T01:19:55.150061+0000) 2023-06-23T01:20:07.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:06 smithi157 conmon[145357]: 2023-06-23T01:20:06.674+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2023-06-23T01:19:36.850984+0000 front 2023-06-23T01:19:36.850994+0000 (oldest deadline 2023-06-23T01:20:02.751055+0000) 2023-06-23T01:20:07.260 INFO:journalctl@ceph.osd.9.smithi120.stdout:Jun 23 01:20:06 smithi120 bash[129772]: ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c-osd-9 2023-06-23T01:20:07.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:07 smithi134 conmon[143713]: 2023-06-23T01:20:07.104+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:20:07.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:07 smithi134 conmon[143713]: 2023-06-23T01:20:07.104+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:20:07.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:07 smithi134 conmon[143713]: 2023-06-23T01:20:07.104+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:20:07.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:07 smithi134 conmon[143713]: 2023-06-23T01:20:07.104+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:17.389622+0000 front 2023-06-23T01:19:17.389746+0000 (oldest deadline 2023-06-23T01:19:42.089671+0000) 2023-06-23T01:20:07.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:07 smithi134 conmon[143713]: 2023-06-23T01:20:07.104+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:22.090003+0000 front 2023-06-23T01:19:22.090048+0000 (oldest deadline 2023-06-23T01:19:47.390149+0000) 2023-06-23T01:20:07.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:07 smithi134 conmon[143713]: 2023-06-23T01:20:07.104+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:31.391457+0000 front 2023-06-23T01:19:31.391562+0000 (oldest deadline 2023-06-23T01:19:55.391959+0000) 2023-06-23T01:20:07.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:07 smithi134 conmon[143713]: 2023-06-23T01:20:07.104+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2023-06-23T01:19:41.293079+0000 front 2023-06-23T01:19:41.293164+0000 (oldest deadline 2023-06-23T01:20:05.992903+0000) 2023-06-23T01:20:08.126 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:07 smithi157 conmon[145357]: 2023-06-23T01:20:07.649+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:20:08.126 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:07 smithi157 conmon[145357]: 2023-06-23T01:20:07.649+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:20:08.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:07 smithi157 conmon[145357]: 2023-06-23T01:20:07.649+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:20:08.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:07 smithi157 conmon[145357]: 2023-06-23T01:20:07.649+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.749536+0000 front 2023-06-23T01:19:25.749472+0000 (oldest deadline 2023-06-23T01:19:49.849573+0000) 2023-06-23T01:20:08.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:07 smithi157 conmon[145357]: 2023-06-23T01:20:07.649+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:29.849868+0000 front 2023-06-23T01:19:29.849894+0000 (oldest deadline 2023-06-23T01:19:55.150061+0000) 2023-06-23T01:20:08.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:07 smithi157 conmon[145357]: 2023-06-23T01:20:07.649+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2023-06-23T01:19:36.850984+0000 front 2023-06-23T01:19:36.850994+0000 (oldest deadline 2023-06-23T01:20:02.751055+0000) 2023-06-23T01:20:08.443 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:08 smithi134 conmon[143713]: 2023-06-23T01:20:08.067+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:20:08.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:08 smithi134 conmon[143713]: 2023-06-23T01:20:08.067+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:20:08.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:08 smithi134 conmon[143713]: 2023-06-23T01:20:08.067+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:20:08.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:08 smithi134 conmon[143713]: 2023-06-23T01:20:08.067+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:17.389622+0000 front 2023-06-23T01:19:17.389746+0000 (oldest deadline 2023-06-23T01:19:42.089671+0000) 2023-06-23T01:20:08.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:08 smithi134 conmon[143713]: 2023-06-23T01:20:08.067+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:22.090003+0000 front 2023-06-23T01:19:22.090048+0000 (oldest deadline 2023-06-23T01:19:47.390149+0000) 2023-06-23T01:20:08.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:08 smithi134 conmon[143713]: 2023-06-23T01:20:08.067+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:31.391457+0000 front 2023-06-23T01:19:31.391562+0000 (oldest deadline 2023-06-23T01:19:55.391959+0000) 2023-06-23T01:20:08.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:08 smithi134 conmon[143713]: 2023-06-23T01:20:08.067+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2023-06-23T01:19:41.293079+0000 front 2023-06-23T01:19:41.293164+0000 (oldest deadline 2023-06-23T01:20:05.992903+0000) 2023-06-23T01:20:08.798 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@osd.9.service' 2023-06-23T01:20:08.879 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T01:20:08.879 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-06-23T01:20:08.880 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-06-23T01:20:08.880 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@osd.10 2023-06-23T01:20:09.126 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:08 smithi157 conmon[145357]: 2023-06-23T01:20:08.645+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:20:09.126 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:08 smithi157 conmon[145357]: 2023-06-23T01:20:08.645+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:20:09.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:08 smithi157 conmon[145357]: 2023-06-23T01:20:08.645+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:20:09.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:08 smithi157 conmon[145357]: 2023-06-23T01:20:08.645+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.749536+0000 front 2023-06-23T01:19:25.749472+0000 (oldest deadline 2023-06-23T01:19:49.849573+0000) 2023-06-23T01:20:09.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:08 smithi157 conmon[145357]: 2023-06-23T01:20:08.645+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:29.849868+0000 front 2023-06-23T01:19:29.849894+0000 (oldest deadline 2023-06-23T01:19:55.150061+0000) 2023-06-23T01:20:09.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:08 smithi157 conmon[145357]: 2023-06-23T01:20:08.645+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2023-06-23T01:19:36.850984+0000 front 2023-06-23T01:19:36.850994+0000 (oldest deadline 2023-06-23T01:20:02.751055+0000) 2023-06-23T01:20:09.193 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:08 smithi134 systemd[1]: Stopping Ceph osd.10 for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:20:09.194 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:09 smithi134 conmon[143713]: 2023-06-23T01:20:09.094+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:20:09.194 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:09 smithi134 conmon[143713]: 2023-06-23T01:20:09.094+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:20:09.194 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:09 smithi134 conmon[143713]: 2023-06-23T01:20:09.094+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:20:09.194 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:09 smithi134 conmon[143713]: 2023-06-23T01:20:09.094+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:17.389622+0000 front 2023-06-23T01:19:17.389746+0000 (oldest deadline 2023-06-23T01:19:42.089671+0000) 2023-06-23T01:20:09.194 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:09 smithi134 conmon[143713]: 2023-06-23T01:20:09.094+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:22.090003+0000 front 2023-06-23T01:19:22.090048+0000 (oldest deadline 2023-06-23T01:19:47.390149+0000) 2023-06-23T01:20:09.195 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:09 smithi134 conmon[143713]: 2023-06-23T01:20:09.094+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:31.391457+0000 front 2023-06-23T01:19:31.391562+0000 (oldest deadline 2023-06-23T01:19:55.391959+0000) 2023-06-23T01:20:09.195 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:09 smithi134 conmon[143713]: 2023-06-23T01:20:09.094+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2023-06-23T01:19:41.293079+0000 front 2023-06-23T01:19:41.293164+0000 (oldest deadline 2023-06-23T01:20:05.992903+0000) 2023-06-23T01:20:09.693 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:09 smithi134 conmon[143713]: 2023-06-23T01:20:09.232+0000 7ffba6c4c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T01:20:09.694 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:09 smithi134 conmon[143713]: 2023-06-23T01:20:09.232+0000 7ffba6c4c700 -1 osd.10 85 *** Got signal Terminated *** 2023-06-23T01:20:09.694 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:09 smithi134 conmon[143713]: 2023-06-23T01:20:09.232+0000 7ffba6c4c700 -1 osd.10 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-23T01:20:10.126 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:09 smithi157 conmon[145357]: 2023-06-23T01:20:09.668+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:20:10.126 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:09 smithi157 conmon[145357]: 2023-06-23T01:20:09.668+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:20:10.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:09 smithi157 conmon[145357]: 2023-06-23T01:20:09.668+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:20:10.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:09 smithi157 conmon[145357]: 2023-06-23T01:20:09.668+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.749536+0000 front 2023-06-23T01:19:25.749472+0000 (oldest deadline 2023-06-23T01:19:49.849573+0000) 2023-06-23T01:20:10.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:09 smithi157 conmon[145357]: 2023-06-23T01:20:09.668+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:29.849868+0000 front 2023-06-23T01:19:29.849894+0000 (oldest deadline 2023-06-23T01:19:55.150061+0000) 2023-06-23T01:20:10.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:09 smithi157 conmon[145357]: 2023-06-23T01:20:09.668+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2023-06-23T01:19:36.850984+0000 front 2023-06-23T01:19:36.850994+0000 (oldest deadline 2023-06-23T01:20:02.751055+0000) 2023-06-23T01:20:10.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:10 smithi134 conmon[143713]: 2023-06-23T01:20:10.119+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:20:10.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:10 smithi134 conmon[143713]: 2023-06-23T01:20:10.119+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:20:10.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:10 smithi134 conmon[143713]: 2023-06-23T01:20:10.119+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:20:10.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:10 smithi134 conmon[143713]: 2023-06-23T01:20:10.119+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:17.389622+0000 front 2023-06-23T01:19:17.389746+0000 (oldest deadline 2023-06-23T01:19:42.089671+0000) 2023-06-23T01:20:10.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:10 smithi134 conmon[143713]: 2023-06-23T01:20:10.119+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:22.090003+0000 front 2023-06-23T01:19:22.090048+0000 (oldest deadline 2023-06-23T01:19:47.390149+0000) 2023-06-23T01:20:10.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:10 smithi134 conmon[143713]: 2023-06-23T01:20:10.119+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:31.391457+0000 front 2023-06-23T01:19:31.391562+0000 (oldest deadline 2023-06-23T01:19:55.391959+0000) 2023-06-23T01:20:10.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:10 smithi134 conmon[143713]: 2023-06-23T01:20:10.119+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2023-06-23T01:19:41.293079+0000 front 2023-06-23T01:19:41.293164+0000 (oldest deadline 2023-06-23T01:20:05.992903+0000) 2023-06-23T01:20:11.126 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:10 smithi157 conmon[145357]: 2023-06-23T01:20:10.649+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:20:11.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:10 smithi157 conmon[145357]: 2023-06-23T01:20:10.649+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:20:11.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:10 smithi157 conmon[145357]: 2023-06-23T01:20:10.649+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:20:11.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:10 smithi157 conmon[145357]: 2023-06-23T01:20:10.649+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.749536+0000 front 2023-06-23T01:19:25.749472+0000 (oldest deadline 2023-06-23T01:19:49.849573+0000) 2023-06-23T01:20:11.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:10 smithi157 conmon[145357]: 2023-06-23T01:20:10.649+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:29.849868+0000 front 2023-06-23T01:19:29.849894+0000 (oldest deadline 2023-06-23T01:19:55.150061+0000) 2023-06-23T01:20:11.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:10 smithi157 conmon[145357]: 2023-06-23T01:20:10.649+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2023-06-23T01:19:36.850984+0000 front 2023-06-23T01:19:36.850994+0000 (oldest deadline 2023-06-23T01:20:02.751055+0000) 2023-06-23T01:20:11.443 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:11 smithi134 conmon[143713]: 2023-06-23T01:20:11.127+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:20:11.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:11 smithi134 conmon[143713]: 2023-06-23T01:20:11.127+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:20:11.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:11 smithi134 conmon[143713]: 2023-06-23T01:20:11.127+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:20:11.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:11 smithi134 conmon[143713]: 2023-06-23T01:20:11.127+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:17.389622+0000 front 2023-06-23T01:19:17.389746+0000 (oldest deadline 2023-06-23T01:19:42.089671+0000) 2023-06-23T01:20:11.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:11 smithi134 conmon[143713]: 2023-06-23T01:20:11.127+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:22.090003+0000 front 2023-06-23T01:19:22.090048+0000 (oldest deadline 2023-06-23T01:19:47.390149+0000) 2023-06-23T01:20:11.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:11 smithi134 conmon[143713]: 2023-06-23T01:20:11.127+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:31.391457+0000 front 2023-06-23T01:19:31.391562+0000 (oldest deadline 2023-06-23T01:19:55.391959+0000) 2023-06-23T01:20:11.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:11 smithi134 conmon[143713]: 2023-06-23T01:20:11.127+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2023-06-23T01:19:41.293079+0000 front 2023-06-23T01:19:41.293164+0000 (oldest deadline 2023-06-23T01:20:05.992903+0000) 2023-06-23T01:20:12.126 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:11 smithi157 conmon[145357]: 2023-06-23T01:20:11.676+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:20:12.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:11 smithi157 conmon[145357]: 2023-06-23T01:20:11.676+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:20:12.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:11 smithi157 conmon[145357]: 2023-06-23T01:20:11.676+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:20:12.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:11 smithi157 conmon[145357]: 2023-06-23T01:20:11.676+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.749536+0000 front 2023-06-23T01:19:25.749472+0000 (oldest deadline 2023-06-23T01:19:49.849573+0000) 2023-06-23T01:20:12.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:11 smithi157 conmon[145357]: 2023-06-23T01:20:11.676+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:29.849868+0000 front 2023-06-23T01:19:29.849894+0000 (oldest deadline 2023-06-23T01:19:55.150061+0000) 2023-06-23T01:20:12.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:11 smithi157 conmon[145357]: 2023-06-23T01:20:11.676+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2023-06-23T01:19:36.850984+0000 front 2023-06-23T01:19:36.850994+0000 (oldest deadline 2023-06-23T01:20:02.751055+0000) 2023-06-23T01:20:12.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:11 smithi157 conmon[145357]: 2023-06-23T01:20:11.676+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2023-06-23T01:19:49.752830+0000 front 2023-06-23T01:19:49.752730+0000 (oldest deadline 2023-06-23T01:20:11.452734+0000) 2023-06-23T01:20:12.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:12 smithi134 conmon[143713]: 2023-06-23T01:20:12.121+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:20:12.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:12 smithi134 conmon[143713]: 2023-06-23T01:20:12.121+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:20:12.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:12 smithi134 conmon[143713]: 2023-06-23T01:20:12.121+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:20:12.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:12 smithi134 conmon[143713]: 2023-06-23T01:20:12.121+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:17.389622+0000 front 2023-06-23T01:19:17.389746+0000 (oldest deadline 2023-06-23T01:19:42.089671+0000) 2023-06-23T01:20:12.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:12 smithi134 conmon[143713]: 2023-06-23T01:20:12.121+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:22.090003+0000 front 2023-06-23T01:19:22.090048+0000 (oldest deadline 2023-06-23T01:19:47.390149+0000) 2023-06-23T01:20:12.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:12 smithi134 conmon[143713]: 2023-06-23T01:20:12.121+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:31.391457+0000 front 2023-06-23T01:19:31.391562+0000 (oldest deadline 2023-06-23T01:19:55.391959+0000) 2023-06-23T01:20:12.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:12 smithi134 conmon[143713]: 2023-06-23T01:20:12.121+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2023-06-23T01:19:41.293079+0000 front 2023-06-23T01:19:41.293164+0000 (oldest deadline 2023-06-23T01:20:05.992903+0000) 2023-06-23T01:20:13.126 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:12 smithi157 conmon[145357]: 2023-06-23T01:20:12.681+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:20:13.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:12 smithi157 conmon[145357]: 2023-06-23T01:20:12.681+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:20:13.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:12 smithi157 conmon[145357]: 2023-06-23T01:20:12.681+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:20:13.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:12 smithi157 conmon[145357]: 2023-06-23T01:20:12.681+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.749536+0000 front 2023-06-23T01:19:25.749472+0000 (oldest deadline 2023-06-23T01:19:49.849573+0000) 2023-06-23T01:20:13.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:12 smithi157 conmon[145357]: 2023-06-23T01:20:12.681+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:29.849868+0000 front 2023-06-23T01:19:29.849894+0000 (oldest deadline 2023-06-23T01:19:55.150061+0000) 2023-06-23T01:20:13.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:12 smithi157 conmon[145357]: 2023-06-23T01:20:12.681+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2023-06-23T01:19:36.850984+0000 front 2023-06-23T01:19:36.850994+0000 (oldest deadline 2023-06-23T01:20:02.751055+0000) 2023-06-23T01:20:13.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:12 smithi157 conmon[145357]: 2023-06-23T01:20:12.681+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2023-06-23T01:19:49.752830+0000 front 2023-06-23T01:19:49.752730+0000 (oldest deadline 2023-06-23T01:20:11.452734+0000) 2023-06-23T01:20:13.443 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:13 smithi134 conmon[143713]: 2023-06-23T01:20:13.154+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:20:13.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:13 smithi134 conmon[143713]: 2023-06-23T01:20:13.154+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:20:13.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:13 smithi134 conmon[143713]: 2023-06-23T01:20:13.154+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:20:13.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:13 smithi134 conmon[143713]: 2023-06-23T01:20:13.154+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:17.389622+0000 front 2023-06-23T01:19:17.389746+0000 (oldest deadline 2023-06-23T01:19:42.089671+0000) 2023-06-23T01:20:13.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:13 smithi134 conmon[143713]: 2023-06-23T01:20:13.154+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:22.090003+0000 front 2023-06-23T01:19:22.090048+0000 (oldest deadline 2023-06-23T01:19:47.390149+0000) 2023-06-23T01:20:13.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:13 smithi134 conmon[143713]: 2023-06-23T01:20:13.154+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:31.391457+0000 front 2023-06-23T01:19:31.391562+0000 (oldest deadline 2023-06-23T01:19:55.391959+0000) 2023-06-23T01:20:13.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:13 smithi134 conmon[143713]: 2023-06-23T01:20:13.154+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2023-06-23T01:19:41.293079+0000 front 2023-06-23T01:19:41.293164+0000 (oldest deadline 2023-06-23T01:20:05.992903+0000) 2023-06-23T01:20:14.126 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:13 smithi157 conmon[145357]: 2023-06-23T01:20:13.674+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:20:14.126 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:13 smithi157 conmon[145357]: 2023-06-23T01:20:13.674+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:20:14.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:13 smithi157 conmon[145357]: 2023-06-23T01:20:13.674+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:20:14.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:13 smithi157 conmon[145357]: 2023-06-23T01:20:13.674+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.749536+0000 front 2023-06-23T01:19:25.749472+0000 (oldest deadline 2023-06-23T01:19:49.849573+0000) 2023-06-23T01:20:14.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:13 smithi157 conmon[145357]: 2023-06-23T01:20:13.674+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:29.849868+0000 front 2023-06-23T01:19:29.849894+0000 (oldest deadline 2023-06-23T01:19:55.150061+0000) 2023-06-23T01:20:14.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:13 smithi157 conmon[145357]: 2023-06-23T01:20:13.674+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2023-06-23T01:19:36.850984+0000 front 2023-06-23T01:19:36.850994+0000 (oldest deadline 2023-06-23T01:20:02.751055+0000) 2023-06-23T01:20:14.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:13 smithi157 conmon[145357]: 2023-06-23T01:20:13.674+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2023-06-23T01:19:49.752830+0000 front 2023-06-23T01:19:49.752730+0000 (oldest deadline 2023-06-23T01:20:11.452734+0000) 2023-06-23T01:20:14.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:14 smithi134 conmon[143713]: 2023-06-23T01:20:14.174+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:51.086010+0000 front 2023-06-23T01:18:51.086059+0000 (oldest deadline 2023-06-23T01:19:16.985760+0000) 2023-06-23T01:20:14.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:14 smithi134 conmon[143713]: 2023-06-23T01:20:14.174+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:18:58.686654+0000 front 2023-06-23T01:18:58.686634+0000 (oldest deadline 2023-06-23T01:19:22.786778+0000) 2023-06-23T01:20:14.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:14 smithi134 conmon[143713]: 2023-06-23T01:20:14.174+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:07.487822+0000 front 2023-06-23T01:19:07.488043+0000 (oldest deadline 2023-06-23T01:19:32.787892+0000) 2023-06-23T01:20:14.444 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:14 smithi134 conmon[143713]: 2023-06-23T01:20:14.174+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2023-06-23T01:19:17.389622+0000 front 2023-06-23T01:19:17.389746+0000 (oldest deadline 2023-06-23T01:19:42.089671+0000) 2023-06-23T01:20:14.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:14 smithi134 conmon[143713]: 2023-06-23T01:20:14.174+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:22.090003+0000 front 2023-06-23T01:19:22.090048+0000 (oldest deadline 2023-06-23T01:19:47.390149+0000) 2023-06-23T01:20:14.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:14 smithi134 conmon[143713]: 2023-06-23T01:20:14.174+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:31.391457+0000 front 2023-06-23T01:19:31.391562+0000 (oldest deadline 2023-06-23T01:19:55.391959+0000) 2023-06-23T01:20:14.445 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:14 smithi134 conmon[143713]: 2023-06-23T01:20:14.174+0000 7ffba324a700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2023-06-23T01:19:41.293079+0000 front 2023-06-23T01:19:41.293164+0000 (oldest deadline 2023-06-23T01:20:05.992903+0000) 2023-06-23T01:20:15.126 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:14 smithi157 conmon[145357]: 2023-06-23T01:20:14.672+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:20:15.126 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:14 smithi157 conmon[145357]: 2023-06-23T01:20:14.672+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:20:15.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:14 smithi157 conmon[145357]: 2023-06-23T01:20:14.672+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:20:15.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:14 smithi157 conmon[145357]: 2023-06-23T01:20:14.672+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.749536+0000 front 2023-06-23T01:19:25.749472+0000 (oldest deadline 2023-06-23T01:19:49.849573+0000) 2023-06-23T01:20:15.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:14 smithi157 conmon[145357]: 2023-06-23T01:20:14.672+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:29.849868+0000 front 2023-06-23T01:19:29.849894+0000 (oldest deadline 2023-06-23T01:19:55.150061+0000) 2023-06-23T01:20:15.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:14 smithi157 conmon[145357]: 2023-06-23T01:20:14.672+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2023-06-23T01:19:36.850984+0000 front 2023-06-23T01:19:36.850994+0000 (oldest deadline 2023-06-23T01:20:02.751055+0000) 2023-06-23T01:20:15.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:14 smithi157 conmon[145357]: 2023-06-23T01:20:14.672+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2023-06-23T01:19:49.752830+0000 front 2023-06-23T01:19:49.752730+0000 (oldest deadline 2023-06-23T01:20:11.452734+0000) 2023-06-23T01:20:15.193 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jun 23 01:20:14 smithi134 bash[153470]: ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c-osd-10 2023-06-23T01:20:16.126 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:15 smithi157 conmon[145357]: 2023-06-23T01:20:15.710+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:20:16.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:15 smithi157 conmon[145357]: 2023-06-23T01:20:15.710+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:20:16.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:15 smithi157 conmon[145357]: 2023-06-23T01:20:15.710+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:20:16.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:15 smithi157 conmon[145357]: 2023-06-23T01:20:15.710+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.749536+0000 front 2023-06-23T01:19:25.749472+0000 (oldest deadline 2023-06-23T01:19:49.849573+0000) 2023-06-23T01:20:16.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:15 smithi157 conmon[145357]: 2023-06-23T01:20:15.710+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:29.849868+0000 front 2023-06-23T01:19:29.849894+0000 (oldest deadline 2023-06-23T01:19:55.150061+0000) 2023-06-23T01:20:16.128 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:15 smithi157 conmon[145357]: 2023-06-23T01:20:15.710+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2023-06-23T01:19:36.850984+0000 front 2023-06-23T01:19:36.850994+0000 (oldest deadline 2023-06-23T01:20:02.751055+0000) 2023-06-23T01:20:16.128 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:15 smithi157 conmon[145357]: 2023-06-23T01:20:15.710+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2023-06-23T01:19:49.752830+0000 front 2023-06-23T01:19:49.752730+0000 (oldest deadline 2023-06-23T01:20:11.452734+0000) 2023-06-23T01:20:16.524 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@osd.10.service' 2023-06-23T01:20:16.562 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T01:20:16.562 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-06-23T01:20:16.563 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-06-23T01:20:16.563 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@osd.11 2023-06-23T01:20:16.876 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:16 smithi157 systemd[1]: Stopping Ceph osd.11 for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:20:16.876 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:16 smithi157 conmon[145357]: 2023-06-23T01:20:16.757+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:20:16.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:16 smithi157 conmon[145357]: 2023-06-23T01:20:16.757+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:20:16.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:16 smithi157 conmon[145357]: 2023-06-23T01:20:16.757+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:20:16.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:16 smithi157 conmon[145357]: 2023-06-23T01:20:16.757+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.749536+0000 front 2023-06-23T01:19:25.749472+0000 (oldest deadline 2023-06-23T01:19:49.849573+0000) 2023-06-23T01:20:16.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:16 smithi157 conmon[145357]: 2023-06-23T01:20:16.757+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:29.849868+0000 front 2023-06-23T01:19:29.849894+0000 (oldest deadline 2023-06-23T01:19:55.150061+0000) 2023-06-23T01:20:16.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:16 smithi157 conmon[145357]: 2023-06-23T01:20:16.757+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2023-06-23T01:19:36.850984+0000 front 2023-06-23T01:19:36.850994+0000 (oldest deadline 2023-06-23T01:20:02.751055+0000) 2023-06-23T01:20:16.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:16 smithi157 conmon[145357]: 2023-06-23T01:20:16.757+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2023-06-23T01:19:49.752830+0000 front 2023-06-23T01:19:49.752730+0000 (oldest deadline 2023-06-23T01:20:11.452734+0000) 2023-06-23T01:20:17.376 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:16 smithi157 conmon[145357]: 2023-06-23T01:20:16.921+0000 7f4e0794a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T01:20:17.377 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:16 smithi157 conmon[145357]: 2023-06-23T01:20:16.921+0000 7f4e0794a700 -1 osd.11 85 *** Got signal Terminated *** 2023-06-23T01:20:17.377 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:16 smithi157 conmon[145357]: 2023-06-23T01:20:16.921+0000 7f4e0794a700 -1 osd.11 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-23T01:20:18.126 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:17 smithi157 conmon[145357]: 2023-06-23T01:20:17.719+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:20:18.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:17 smithi157 conmon[145357]: 2023-06-23T01:20:17.719+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:20:18.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:17 smithi157 conmon[145357]: 2023-06-23T01:20:17.719+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:20:18.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:17 smithi157 conmon[145357]: 2023-06-23T01:20:17.719+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.749536+0000 front 2023-06-23T01:19:25.749472+0000 (oldest deadline 2023-06-23T01:19:49.849573+0000) 2023-06-23T01:20:18.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:17 smithi157 conmon[145357]: 2023-06-23T01:20:17.719+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:29.849868+0000 front 2023-06-23T01:19:29.849894+0000 (oldest deadline 2023-06-23T01:19:55.150061+0000) 2023-06-23T01:20:18.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:17 smithi157 conmon[145357]: 2023-06-23T01:20:17.719+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2023-06-23T01:19:36.850984+0000 front 2023-06-23T01:19:36.850994+0000 (oldest deadline 2023-06-23T01:20:02.751055+0000) 2023-06-23T01:20:18.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:17 smithi157 conmon[145357]: 2023-06-23T01:20:17.719+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2023-06-23T01:19:49.752830+0000 front 2023-06-23T01:19:49.752730+0000 (oldest deadline 2023-06-23T01:20:11.452734+0000) 2023-06-23T01:20:19.126 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:18 smithi157 conmon[145357]: 2023-06-23T01:20:18.742+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:20:19.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:18 smithi157 conmon[145357]: 2023-06-23T01:20:18.742+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:20:19.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:18 smithi157 conmon[145357]: 2023-06-23T01:20:18.742+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:20:19.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:18 smithi157 conmon[145357]: 2023-06-23T01:20:18.742+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.749536+0000 front 2023-06-23T01:19:25.749472+0000 (oldest deadline 2023-06-23T01:19:49.849573+0000) 2023-06-23T01:20:19.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:18 smithi157 conmon[145357]: 2023-06-23T01:20:18.742+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:29.849868+0000 front 2023-06-23T01:19:29.849894+0000 (oldest deadline 2023-06-23T01:19:55.150061+0000) 2023-06-23T01:20:19.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:18 smithi157 conmon[145357]: 2023-06-23T01:20:18.742+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2023-06-23T01:19:36.850984+0000 front 2023-06-23T01:19:36.850994+0000 (oldest deadline 2023-06-23T01:20:02.751055+0000) 2023-06-23T01:20:19.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:18 smithi157 conmon[145357]: 2023-06-23T01:20:18.742+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2023-06-23T01:19:49.752830+0000 front 2023-06-23T01:19:49.752730+0000 (oldest deadline 2023-06-23T01:20:11.452734+0000) 2023-06-23T01:20:20.126 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:19 smithi157 conmon[145357]: 2023-06-23T01:20:19.742+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:20:20.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:19 smithi157 conmon[145357]: 2023-06-23T01:20:19.742+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:20:20.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:19 smithi157 conmon[145357]: 2023-06-23T01:20:19.742+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:20:20.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:19 smithi157 conmon[145357]: 2023-06-23T01:20:19.742+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.749536+0000 front 2023-06-23T01:19:25.749472+0000 (oldest deadline 2023-06-23T01:19:49.849573+0000) 2023-06-23T01:20:20.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:19 smithi157 conmon[145357]: 2023-06-23T01:20:19.742+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:29.849868+0000 front 2023-06-23T01:19:29.849894+0000 (oldest deadline 2023-06-23T01:19:55.150061+0000) 2023-06-23T01:20:20.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:19 smithi157 conmon[145357]: 2023-06-23T01:20:19.742+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2023-06-23T01:19:36.850984+0000 front 2023-06-23T01:19:36.850994+0000 (oldest deadline 2023-06-23T01:20:02.751055+0000) 2023-06-23T01:20:20.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:19 smithi157 conmon[145357]: 2023-06-23T01:20:19.742+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2023-06-23T01:19:49.752830+0000 front 2023-06-23T01:19:49.752730+0000 (oldest deadline 2023-06-23T01:20:11.452734+0000) 2023-06-23T01:20:21.126 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:20 smithi157 conmon[145357]: 2023-06-23T01:20:20.759+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:20:21.126 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:20 smithi157 conmon[145357]: 2023-06-23T01:20:20.759+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:20:21.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:20 smithi157 conmon[145357]: 2023-06-23T01:20:20.759+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:20:21.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:20 smithi157 conmon[145357]: 2023-06-23T01:20:20.759+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.749536+0000 front 2023-06-23T01:19:25.749472+0000 (oldest deadline 2023-06-23T01:19:49.849573+0000) 2023-06-23T01:20:21.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:20 smithi157 conmon[145357]: 2023-06-23T01:20:20.759+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:29.849868+0000 front 2023-06-23T01:19:29.849894+0000 (oldest deadline 2023-06-23T01:19:55.150061+0000) 2023-06-23T01:20:21.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:20 smithi157 conmon[145357]: 2023-06-23T01:20:20.759+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2023-06-23T01:19:36.850984+0000 front 2023-06-23T01:19:36.850994+0000 (oldest deadline 2023-06-23T01:20:02.751055+0000) 2023-06-23T01:20:21.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:20 smithi157 conmon[145357]: 2023-06-23T01:20:20.759+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2023-06-23T01:19:49.752830+0000 front 2023-06-23T01:19:49.752730+0000 (oldest deadline 2023-06-23T01:20:11.452734+0000) 2023-06-23T01:20:22.126 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:21 smithi157 conmon[145357]: 2023-06-23T01:20:21.722+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2023-06-23T01:18:50.943015+0000 front 2023-06-23T01:18:50.942944+0000 (oldest deadline 2023-06-23T01:19:14.442793+0000) 2023-06-23T01:20:22.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:21 smithi157 conmon[145357]: 2023-06-23T01:20:21.722+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-06-23T01:19:00.145113+0000 front 2023-06-23T01:19:00.145155+0000 (oldest deadline 2023-06-23T01:19:22.445028+0000) 2023-06-23T01:20:22.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:21 smithi157 conmon[145357]: 2023-06-23T01:20:21.722+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2023-06-23T01:19:08.246523+0000 front 2023-06-23T01:19:08.246458+0000 (oldest deadline 2023-06-23T01:19:33.546639+0000) 2023-06-23T01:20:22.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:21 smithi157 conmon[145357]: 2023-06-23T01:20:21.722+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-06-23T01:19:25.749536+0000 front 2023-06-23T01:19:25.749472+0000 (oldest deadline 2023-06-23T01:19:49.849573+0000) 2023-06-23T01:20:22.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:21 smithi157 conmon[145357]: 2023-06-23T01:20:21.722+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2023-06-23T01:19:29.849868+0000 front 2023-06-23T01:19:29.849894+0000 (oldest deadline 2023-06-23T01:19:55.150061+0000) 2023-06-23T01:20:22.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:21 smithi157 conmon[145357]: 2023-06-23T01:20:21.722+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2023-06-23T01:19:36.850984+0000 front 2023-06-23T01:19:36.850994+0000 (oldest deadline 2023-06-23T01:20:02.751055+0000) 2023-06-23T01:20:22.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:21 smithi157 conmon[145357]: 2023-06-23T01:20:21.722+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2023-06-23T01:19:49.752830+0000 front 2023-06-23T01:19:49.752730+0000 (oldest deadline 2023-06-23T01:20:11.452734+0000) 2023-06-23T01:20:22.127 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:21 smithi157 conmon[145357]: 2023-06-23T01:20:21.722+0000 7f4e02f46700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6820 osd.8 since back 2023-06-23T01:19:57.353789+0000 front 2023-06-23T01:19:57.353804+0000 (oldest deadline 2023-06-23T01:20:20.853861+0000) 2023-06-23T01:20:22.877 INFO:journalctl@ceph.osd.11.smithi157.stdout:Jun 23 01:20:22 smithi157 bash[155545]: ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c-osd-11 2023-06-23T01:20:24.523 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@osd.11.service' 2023-06-23T01:20:24.561 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T01:20:24.562 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-06-23T01:20:24.563 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-06-23T01:20:24.564 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mds.a 2023-06-23T01:20:25.010 INFO:journalctl@ceph.mds.a.smithi120.stdout:Jun 23 01:20:24 smithi120 systemd[1]: Stopping Ceph mds.a for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:20:25.510 INFO:journalctl@ceph.mds.a.smithi120.stdout:Jun 23 01:20:25 smithi120 conmon[124697]: 2023-06-23T01:20:25.145+0000 7fc239aef700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T01:20:25.510 INFO:journalctl@ceph.mds.a.smithi120.stdout:Jun 23 01:20:25 smithi120 conmon[124697]: 2023-06-23T01:20:25.145+0000 7fc239aef700 -1 mds.a *** got signal Terminated *** 2023-06-23T01:20:29.760 INFO:journalctl@ceph.mds.a.smithi120.stdout:Jun 23 01:20:29 smithi120 bash[130331]: ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c-mds-a 2023-06-23T01:20:29.813 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mds.a.service' 2023-06-23T01:20:29.888 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T01:20:29.888 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-06-23T01:20:29.889 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-06-23T01:20:29.889 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mds.d 2023-06-23T01:20:30.260 INFO:journalctl@ceph.mds.d.smithi120.stdout:Jun 23 01:20:29 smithi120 systemd[1]: Stopping Ceph mds.d for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:20:30.760 INFO:journalctl@ceph.mds.d.smithi120.stdout:Jun 23 01:20:30 smithi120 conmon[123236]: 2023-06-23T01:20:30.370+0000 7ff2b43f8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T01:20:30.760 INFO:journalctl@ceph.mds.d.smithi120.stdout:Jun 23 01:20:30 smithi120 conmon[123236]: 2023-06-23T01:20:30.370+0000 7ff2b43f8700 -1 mds.d *** got signal Terminated *** 2023-06-23T01:20:34.510 INFO:journalctl@ceph.mds.d.smithi120.stdout:Jun 23 01:20:34 smithi120 bash[130608]: ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c-mds-d 2023-06-23T01:20:34.627 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mds.d.service' 2023-06-23T01:20:34.699 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T01:20:34.699 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-06-23T01:20:34.700 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-06-23T01:20:34.700 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mds.g 2023-06-23T01:20:35.010 INFO:journalctl@ceph.mds.g.smithi120.stdout:Jun 23 01:20:34 smithi120 systemd[1]: Stopping Ceph mds.g for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:20:35.510 INFO:journalctl@ceph.mds.g.smithi120.stdout:Jun 23 01:20:35 smithi120 conmon[124089]: 2023-06-23T01:20:35.186+0000 7fee00d65700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T01:20:35.510 INFO:journalctl@ceph.mds.g.smithi120.stdout:Jun 23 01:20:35 smithi120 conmon[124089]: 2023-06-23T01:20:35.186+0000 7fee00d65700 -1 mds.g *** got signal Terminated *** 2023-06-23T01:20:38.510 INFO:journalctl@ceph.mds.g.smithi120.stdout:Jun 23 01:20:38 smithi120 bash[130891]: ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c-mds-g 2023-06-23T01:20:38.721 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mds.g.service' 2023-06-23T01:20:38.793 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T01:20:38.793 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-06-23T01:20:38.794 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-06-23T01:20:38.794 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mds.j 2023-06-23T01:20:39.010 INFO:journalctl@ceph.mds.j.smithi120.stdout:Jun 23 01:20:38 smithi120 systemd[1]: Stopping Ceph mds.j for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:20:39.760 INFO:journalctl@ceph.mds.j.smithi120.stdout:Jun 23 01:20:39 smithi120 conmon[122634]: 2023-06-23T01:20:39.285+0000 7f18d950a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T01:20:39.760 INFO:journalctl@ceph.mds.j.smithi120.stdout:Jun 23 01:20:39 smithi120 conmon[122634]: 2023-06-23T01:20:39.285+0000 7f18d950a700 -1 mds.j *** got signal Terminated *** 2023-06-23T01:20:43.504 INFO:journalctl@ceph.mds.j.smithi120.stdout:Jun 23 01:20:43 smithi120 bash[131177]: ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c-mds-j 2023-06-23T01:20:43.620 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mds.j.service' 2023-06-23T01:20:43.692 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T01:20:43.693 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-06-23T01:20:43.694 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-06-23T01:20:43.694 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mds.b 2023-06-23T01:20:44.057 INFO:journalctl@ceph.mds.b.smithi134.stdout:Jun 23 01:20:43 smithi134 systemd[1]: Stopping Ceph mds.b for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:20:44.443 INFO:journalctl@ceph.mds.b.smithi134.stdout:Jun 23 01:20:44 smithi134 conmon[148932]: 2023-06-23T01:20:44.052+0000 7f92c614d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T01:20:44.444 INFO:journalctl@ceph.mds.b.smithi134.stdout:Jun 23 01:20:44 smithi134 conmon[148932]: 2023-06-23T01:20:44.052+0000 7f92c614d700 -1 mds.b *** got signal Terminated *** 2023-06-23T01:20:49.443 INFO:journalctl@ceph.mds.b.smithi134.stdout:Jun 23 01:20:49 smithi134 bash[154026]: ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c-mds-b 2023-06-23T01:20:49.465 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mds.b.service' 2023-06-23T01:20:49.499 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T01:20:49.500 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-06-23T01:20:49.501 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-06-23T01:20:49.501 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mds.e 2023-06-23T01:20:49.860 INFO:journalctl@ceph.mds.e.smithi134.stdout:Jun 23 01:20:49 smithi134 systemd[1]: Stopping Ceph mds.e for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:20:50.193 INFO:journalctl@ceph.mds.e.smithi134.stdout:Jun 23 01:20:49 smithi134 conmon[147794]: 2023-06-23T01:20:49.856+0000 7fcac0875700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T01:20:50.194 INFO:journalctl@ceph.mds.e.smithi134.stdout:Jun 23 01:20:49 smithi134 conmon[147794]: 2023-06-23T01:20:49.856+0000 7fcac0875700 -1 mds.e *** got signal Terminated *** 2023-06-23T01:20:52.943 INFO:journalctl@ceph.mds.e.smithi134.stdout:Jun 23 01:20:52 smithi134 bash[154308]: ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c-mds-e 2023-06-23T01:20:53.135 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mds.e.service' 2023-06-23T01:20:53.168 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T01:20:53.168 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-06-23T01:20:53.169 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-06-23T01:20:53.169 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mds.h 2023-06-23T01:20:53.443 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jun 23 01:20:53 smithi134 systemd[1]: Stopping Ceph mds.h for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:20:53.944 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jun 23 01:20:53 smithi134 conmon[148364]: 2023-06-23T01:20:53.522+0000 7f6794e65700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T01:20:53.944 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jun 23 01:20:53 smithi134 conmon[148364]: 2023-06-23T01:20:53.522+0000 7f6794e65700 -1 mds.h *** got signal Terminated *** 2023-06-23T01:20:56.654 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jun 23 01:20:56 smithi134 bash[154593]: ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c-mds-h 2023-06-23T01:20:56.735 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mds.h.service' 2023-06-23T01:20:56.765 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T01:20:56.766 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-06-23T01:20:56.766 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-06-23T01:20:56.767 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mds.k 2023-06-23T01:20:57.125 INFO:journalctl@ceph.mds.k.smithi134.stdout:Jun 23 01:20:56 smithi134 systemd[1]: Stopping Ceph mds.k for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:20:57.443 INFO:journalctl@ceph.mds.k.smithi134.stdout:Jun 23 01:20:57 smithi134 conmon[147233]: 2023-06-23T01:20:57.121+0000 7f6768967700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T01:20:57.444 INFO:journalctl@ceph.mds.k.smithi134.stdout:Jun 23 01:20:57 smithi134 conmon[147233]: 2023-06-23T01:20:57.121+0000 7f6768967700 -1 mds.k *** got signal Terminated *** 2023-06-23T01:21:00.443 INFO:journalctl@ceph.mds.k.smithi134.stdout:Jun 23 01:21:00 smithi134 bash[154874]: ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c-mds-k 2023-06-23T01:21:00.563 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mds.k.service' 2023-06-23T01:21:00.593 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T01:21:00.593 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-06-23T01:21:00.594 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-06-23T01:21:00.594 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mds.c 2023-06-23T01:21:00.876 INFO:journalctl@ceph.mds.c.smithi157.stdout:Jun 23 01:21:00 smithi157 systemd[1]: Stopping Ceph mds.c for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:21:01.376 INFO:journalctl@ceph.mds.c.smithi157.stdout:Jun 23 01:21:00 smithi157 conmon[150461]: 2023-06-23T01:21:00.931+0000 7f590c234700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T01:21:01.376 INFO:journalctl@ceph.mds.c.smithi157.stdout:Jun 23 01:21:00 smithi157 conmon[150461]: 2023-06-23T01:21:00.931+0000 7f590c234700 -1 mds.c *** got signal Terminated *** 2023-06-23T01:21:05.543 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mds.c.service' 2023-06-23T01:21:05.577 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T01:21:05.577 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-06-23T01:21:05.578 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-06-23T01:21:05.578 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mds.f 2023-06-23T01:21:05.950 INFO:journalctl@ceph.mds.f.smithi157.stdout:Jun 23 01:21:05 smithi157 systemd[1]: Stopping Ceph mds.f for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:21:06.376 INFO:journalctl@ceph.mds.f.smithi157.stdout:Jun 23 01:21:05 smithi157 conmon[149898]: 2023-06-23T01:21:05.946+0000 7f780243a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T01:21:06.377 INFO:journalctl@ceph.mds.f.smithi157.stdout:Jun 23 01:21:05 smithi157 conmon[149898]: 2023-06-23T01:21:05.946+0000 7f780243a700 -1 mds.f *** got signal Terminated *** 2023-06-23T01:21:07.653 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mds.f.service' 2023-06-23T01:21:07.686 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T01:21:07.687 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-06-23T01:21:07.688 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-06-23T01:21:07.688 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mds.i 2023-06-23T01:21:08.066 INFO:journalctl@ceph.mds.i.smithi157.stdout:Jun 23 01:21:07 smithi157 systemd[1]: Stopping Ceph mds.i for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:21:08.376 INFO:journalctl@ceph.mds.i.smithi157.stdout:Jun 23 01:21:08 smithi157 conmon[151022]: 2023-06-23T01:21:08.062+0000 7f523e21b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T01:21:08.377 INFO:journalctl@ceph.mds.i.smithi157.stdout:Jun 23 01:21:08 smithi157 conmon[151022]: 2023-06-23T01:21:08.062+0000 7f523e21b700 -1 mds.i *** got signal Terminated *** 2023-06-23T01:21:11.127 INFO:journalctl@ceph.mds.i.smithi157.stdout:Jun 23 01:21:10 smithi157 bash[156652]: ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c-mds-i 2023-06-23T01:21:11.295 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mds.i.service' 2023-06-23T01:21:11.327 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T01:21:11.328 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-06-23T01:21:11.329 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-06-23T01:21:11.330 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mds.l 2023-06-23T01:21:11.626 INFO:journalctl@ceph.mds.l.smithi157.stdout:Jun 23 01:21:11 smithi157 systemd[1]: Stopping Ceph mds.l for 9efbfa8e-1162-11ee-9b2b-001a4aab830c... 2023-06-23T01:21:12.126 INFO:journalctl@ceph.mds.l.smithi157.stdout:Jun 23 01:21:11 smithi157 conmon[149327]: 2023-06-23T01:21:11.683+0000 7f4023ad5700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T01:21:12.127 INFO:journalctl@ceph.mds.l.smithi157.stdout:Jun 23 01:21:11 smithi157 conmon[149327]: 2023-06-23T01:21:11.683+0000 7f4023ad5700 -1 mds.l *** got signal Terminated *** 2023-06-23T01:21:13.876 INFO:journalctl@ceph.mds.l.smithi157.stdout:Jun 23 01:21:13 smithi157 bash[156936]: ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c-mds-l 2023-06-23T01:21:14.224 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9efbfa8e-1162-11ee-9b2b-001a4aab830c@mds.l.service' 2023-06-23T01:21:14.255 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T01:21:14.255 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-06-23T01:21:14.256 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c --force --keep-logs 2023-06-23T01:21:18.028 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c --force --keep-logs 2023-06-23T01:21:19.961 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c --force --keep-logs 2023-06-23T01:21:21.920 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-06-23T01:21:21.993 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-06-23T01:21:22.025 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-06-23T01:21:22.053 INFO:tasks.cephadm:Archiving crash dumps... 2023-06-23T01:21:22.060 DEBUG:teuthology.misc:Transferring archived files from smithi120:/var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/crash to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311926/remote/smithi120/crash 2023-06-23T01:21:22.062 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/crash -- . 2023-06-23T01:21:22.136 INFO:teuthology.orchestra.run.smithi120.stderr:tar: /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/crash: Cannot open: No such file or directory 2023-06-23T01:21:22.136 INFO:teuthology.orchestra.run.smithi120.stderr:tar: Error is not recoverable: exiting now 2023-06-23T01:21:22.138 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/crash to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311926/remote/smithi134/crash 2023-06-23T01:21:22.139 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/crash -- . 2023-06-23T01:21:22.180 INFO:teuthology.orchestra.run.smithi134.stderr:tar: /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/crash: Cannot open: No such file or directory 2023-06-23T01:21:22.180 INFO:teuthology.orchestra.run.smithi134.stderr:tar: Error is not recoverable: exiting now 2023-06-23T01:21:22.182 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/crash to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311926/remote/smithi157/crash 2023-06-23T01:21:22.184 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/crash -- . 2023-06-23T01:21:22.226 INFO:teuthology.orchestra.run.smithi157.stderr:tar: /var/lib/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/crash: Cannot open: No such file or directory 2023-06-23T01:21:22.226 INFO:teuthology.orchestra.run.smithi157.stderr:tar: Error is not recoverable: exiting now 2023-06-23T01:21:22.227 INFO:tasks.cephadm:Checking cluster log for badness... 2023-06-23T01:21:22.228 DEBUG:teuthology.orchestra.run.smithi120:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-06-23T01:21:22.297 INFO:teuthology.orchestra.run.smithi120.stderr:grep: /var/log/ceph/9efbfa8e-1162-11ee-9b2b-001a4aab830c/ceph.log: No such file or directory 2023-06-23T01:21:22.327 INFO:tasks.cephadm:Compressing logs... 2023-06-23T01:21:22.327 DEBUG:teuthology.orchestra.run.smithi120:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-06-23T01:21:22.332 DEBUG:teuthology.orchestra.run.smithi134:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-06-23T01:21:22.334 DEBUG:teuthology.orchestra.run.smithi157:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-06-23T01:21:22.358 INFO:teuthology.orchestra.run.smithi157.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-06-23T01:21:22.360 INFO:teuthology.orchestra.run.smithi134.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-06-23T01:21:22.390 INFO:teuthology.orchestra.run.smithi120.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-06-23T01:21:23.235 INFO:tasks.cephadm:Archiving logs... 2023-06-23T01:21:23.236 DEBUG:teuthology.misc:Transferring archived files from smithi120:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311926/remote/smithi120/log 2023-06-23T01:21:23.237 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /var/log/ceph -- . 2023-06-23T01:21:23.429 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311926/remote/smithi134/log 2023-06-23T01:21:23.431 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/log/ceph -- . 2023-06-23T01:21:23.542 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311926/remote/smithi157/log 2023-06-23T01:21:23.545 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/log/ceph -- . 2023-06-23T01:21:23.639 INFO:tasks.cephadm:Removing cluster... 2023-06-23T01:21:23.640 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c --force 2023-06-23T01:21:24.763 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c --force 2023-06-23T01:21:25.543 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9efbfa8e-1162-11ee-9b2b-001a4aab830c --force 2023-06-23T01:21:26.323 INFO:tasks.cephadm:Removing cephadm ... 2023-06-23T01:21:26.324 DEBUG:teuthology.orchestra.run.smithi120:> rm -rf /home/ubuntu/cephtest/cephadm 2023-06-23T01:21:26.365 DEBUG:teuthology.orchestra.run.smithi134:> rm -rf /home/ubuntu/cephtest/cephadm 2023-06-23T01:21:26.383 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf /home/ubuntu/cephtest/cephadm 2023-06-23T01:21:26.399 INFO:tasks.cephadm:Teardown complete 2023-06-23T01:21:26.400 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephadm.py", line 1561, in task with contextutil.nested( File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephadm.py", line 1474, in initialize_config yield File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephadm.py", line 896, in cephfs_setup fs = Filesystem(ctx, fs_config=temp, name=name, create=True) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 514, in __init__ self.create() File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 714, in create self.set_max_mds(max_mds) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 583, in set_max_mds self.set_var("max_mds", "%d" % max_mds) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 574, in set_var self.mon_manager.raw_cluster_cmd("fs", "set", self.name, var, *a) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph_manager.py", line 1624, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph_manager.py", line 1615, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi120 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5' 2023-06-23T01:21:26.482 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=c84e5b4752344dc0b4da7818d9855762 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephadm.py", line 1561, in task with contextutil.nested( File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephadm.py", line 1474, in initialize_config yield File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephadm.py", line 896, in cephfs_setup fs = Filesystem(ctx, fs_config=temp, name=name, create=True) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 514, in __init__ self.create() File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 714, in create self.set_max_mds(max_mds) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 583, in set_max_mds self.set_var("max_mds", "%d" % max_mds) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 574, in set_var self.mon_manager.raw_cluster_cmd("fs", "set", self.name, var, *a) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph_manager.py", line 1624, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph_manager.py", line 1615, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi120 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5' 2023-06-23T01:21:26.488 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-06-23T01:21:26.506 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-06-23T01:21:26.554 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-06-23T01:21:26.555 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-06-23T01:21:26.575 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-06-23T01:21:26.579 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-06-23T01:21:26.730 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-06-23T01:21:26.741 INFO:teuthology.task.clock:Checking final clock skew... 2023-06-23T01:21:26.741 DEBUG:teuthology.orchestra.run.smithi120:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-23T01:21:26.746 DEBUG:teuthology.orchestra.run.smithi134:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-23T01:21:26.749 DEBUG:teuthology.orchestra.run.smithi157:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-23T01:21:26.764 INFO:teuthology.orchestra.run.smithi134.stderr:bash: ntpq: command not found 2023-06-23T01:21:26.765 INFO:teuthology.orchestra.run.smithi157.stderr:bash: ntpq: command not found 2023-06-23T01:21:26.765 INFO:teuthology.orchestra.run.smithi157.stdout:210 Number of sources = 4 2023-06-23T01:21:26.766 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-23T01:21:26.766 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2023-06-23T01:21:26.766 INFO:teuthology.orchestra.run.smithi157.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 15 +2017us[+2017us] +/- 97ms 2023-06-23T01:21:26.766 INFO:teuthology.orchestra.run.smithi157.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 16 -1948us[-1948us] +/- 89ms 2023-06-23T01:21:26.767 INFO:teuthology.orchestra.run.smithi157.stdout:^* hv03.front.sepia.ceph.com 3 6 377 16 -37us[ -37us] +/- 67ms 2023-06-23T01:21:26.767 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T01:21:26.769 INFO:teuthology.orchestra.run.smithi134.stdout:210 Number of sources = 4 2023-06-23T01:21:26.769 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-23T01:21:26.770 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2023-06-23T01:21:26.770 INFO:teuthology.orchestra.run.smithi134.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 15 +2099us[+2124us] +/- 97ms 2023-06-23T01:21:26.770 INFO:teuthology.orchestra.run.smithi134.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 14 -1877us[-1852us] +/- 89ms 2023-06-23T01:21:26.770 INFO:teuthology.orchestra.run.smithi134.stdout:^* hv03.front.sepia.ceph.com 3 6 377 15 +27us[ +52us] +/- 67ms 2023-06-23T01:21:26.771 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T01:21:26.778 INFO:teuthology.orchestra.run.smithi120.stderr:bash: ntpq: command not found 2023-06-23T01:21:26.785 INFO:teuthology.orchestra.run.smithi120.stdout:210 Number of sources = 4 2023-06-23T01:21:26.786 INFO:teuthology.orchestra.run.smithi120.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-23T01:21:26.787 INFO:teuthology.orchestra.run.smithi120.stdout:=============================================================================== 2023-06-23T01:21:26.787 INFO:teuthology.orchestra.run.smithi120.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 16 +2127us[+2132us] +/- 97ms 2023-06-23T01:21:26.787 INFO:teuthology.orchestra.run.smithi120.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 16 -1849us[-1844us] +/- 89ms 2023-06-23T01:21:26.787 INFO:teuthology.orchestra.run.smithi120.stdout:^* hv03.front.sepia.ceph.com 3 6 377 15 +60us[ +64us] +/- 67ms 2023-06-23T01:21:26.788 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T01:21:26.789 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-06-23T01:21:26.811 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-06-23T01:21:26.813 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-06-23T01:21:26.835 DEBUG:teuthology.orchestra.run.smithi120:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-06-23T01:21:26.994 DEBUG:teuthology.orchestra.run.smithi134:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-06-23T01:21:27.094 DEBUG:teuthology.orchestra.run.smithi157:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-06-23T01:21:27.187 DEBUG:teuthology.orchestra.run.smithi120:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-23T01:21:27.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T01:21:27.257 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-23T01:21:27.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T01:21:27.289 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-23T01:21:27.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T01:21:27.319 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-06-23T01:21:27.331 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-06-23T01:21:27.379 INFO:teuthology.task.internal:Duration was 1609.731477 seconds 2023-06-23T01:21:27.380 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-06-23T01:21:27.429 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-06-23T01:21:27.594 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-06-23T01:21:27.599 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-06-23T01:21:27.601 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-06-23T01:21:27.645 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-23T01:21:27.645 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-23T01:21:27.703 INFO:teuthology.orchestra.run.smithi120.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-23T01:21:28.037 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-06-23T01:21:28.037 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi120.front.sepia.ceph.com 2023-06-23T01:21:28.038 DEBUG:teuthology.orchestra.run.smithi120:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-06-23T01:21:28.088 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi134.front.sepia.ceph.com 2023-06-23T01:21:28.089 DEBUG:teuthology.orchestra.run.smithi134:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-06-23T01:21:28.109 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi157.front.sepia.ceph.com 2023-06-23T01:21:28.110 DEBUG:teuthology.orchestra.run.smithi157:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-06-23T01:21:28.130 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-06-23T01:21:28.131 DEBUG:teuthology.orchestra.run.smithi120:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-06-23T01:21:28.137 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-06-23T01:21:28.152 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-06-23T01:21:28.217 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-06-23T01:21:28.217 DEBUG:teuthology.orchestra.run.smithi120:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-06-23T01:21:28.223 DEBUG:teuthology.orchestra.run.smithi134:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-06-23T01:21:28.237 DEBUG:teuthology.orchestra.run.smithi157:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-06-23T01:21:28.353 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-06-23T01:21:28.367 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-06-23T01:21:28.368 DEBUG:teuthology.orchestra.run.smithi120:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-06-23T01:21:28.429 DEBUG:teuthology.orchestra.run.smithi134:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-06-23T01:21:28.454 DEBUG:teuthology.orchestra.run.smithi157:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-06-23T01:21:28.481 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-06-23T01:21:28.491 DEBUG:teuthology.orchestra.run.smithi120:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-06-23T01:21:28.497 DEBUG:teuthology.orchestra.run.smithi134:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-06-23T01:21:28.500 DEBUG:teuthology.orchestra.run.smithi157:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-06-23T01:21:28.521 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = core 2023-06-23T01:21:28.548 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = core 2023-06-23T01:21:28.552 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern = core 2023-06-23T01:21:28.593 DEBUG:teuthology.orchestra.run.smithi120:> test -e /home/ubuntu/cephtest/archive/coredump 2023-06-23T01:21:28.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T01:21:28.628 DEBUG:teuthology.orchestra.run.smithi134:> test -e /home/ubuntu/cephtest/archive/coredump 2023-06-23T01:21:28.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T01:21:28.643 DEBUG:teuthology.orchestra.run.smithi157:> test -e /home/ubuntu/cephtest/archive/coredump 2023-06-23T01:21:28.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T01:21:28.658 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-06-23T01:21:28.678 INFO:teuthology.task.internal:Transferring archived files... 2023-06-23T01:21:28.679 DEBUG:teuthology.misc:Transferring archived files from smithi120:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311926/remote/smithi120 2023-06-23T01:21:28.680 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-06-23T01:21:28.752 DEBUG:teuthology.misc:Transferring archived files from smithi134:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311926/remote/smithi134 2023-06-23T01:21:28.752 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-06-23T01:21:28.795 DEBUG:teuthology.misc:Transferring archived files from smithi157:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311926/remote/smithi157 2023-06-23T01:21:28.796 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-06-23T01:21:28.848 INFO:teuthology.task.internal:Removing archive directory... 2023-06-23T01:21:28.849 DEBUG:teuthology.orchestra.run.smithi120:> rm -rf -- /home/ubuntu/cephtest/archive 2023-06-23T01:21:28.854 DEBUG:teuthology.orchestra.run.smithi134:> rm -rf -- /home/ubuntu/cephtest/archive 2023-06-23T01:21:28.856 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf -- /home/ubuntu/cephtest/archive 2023-06-23T01:21:28.885 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-06-23T01:21:28.912 INFO:teuthology.task.internal:Not uploading archives. 2023-06-23T01:21:28.913 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-06-23T01:21:28.934 INFO:teuthology.task.internal:Tidying up after the test... 2023-06-23T01:21:28.935 DEBUG:teuthology.orchestra.run.smithi120:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-06-23T01:21:28.941 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-06-23T01:21:28.945 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-06-23T01:21:28.956 INFO:teuthology.orchestra.run.smithi134.stdout: 262149 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 23 01:21 /home/ubuntu/cephtest 2023-06-23T01:21:28.959 INFO:teuthology.orchestra.run.smithi157.stdout: 262149 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 23 01:21 /home/ubuntu/cephtest 2023-06-23T01:21:28.972 INFO:teuthology.orchestra.run.smithi120.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 23 01:21 /home/ubuntu/cephtest 2023-06-23T01:21:28.978 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-06-23T01:21:29.003 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-06-23T01:21:29.056 INFO:teuthology.nuke:Checking targets against current locks 2023-06-23T01:21:29.087 DEBUG:teuthology.nuke:shortname: smithi120 2023-06-23T01:21:29.088 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-23T01:21:29.116 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi120.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311926', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-23 00:45:09.815243', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ6Gw4c6Wuh55BSbPgPQzg/VvHfenNu8gJCtRHYvUY/SLgFHx7pJ5QOw8bw781Efgu+44roU0rYFUEoea+siels='} 2023-06-23T01:21:29.122 DEBUG:teuthology.nuke:shortname: smithi134 2023-06-23T01:21:29.123 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-23T01:21:29.153 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi134.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311926', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-23 00:45:09.817576', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCqbMNWpk8JK7EEWwa+i86XieFoQ7Oz6383Of1HxWdKzciwj65oZ3grvieXdPsQrjNXLSskieDgL+TrdgNVRy5kSxiX21olKG6XTgbxWbw1/6NsLbnXMMbKPmzjrlDKiTPhpbN9myR4DZyMBwSJlA8oYqnXdBWEcxxNV7M0NT6YBz5P2WS2/r4N6bvAnojjMc95dGH6pq40lg1PDn2qPwd+mEJolkRUr81sdxJhnQCADb54k6R8Wi6nVokYn5UIOPMoKCXJfx1cMxWIugivkdK2n1XUPDdyY7Dk1tU9rifx4QPgnB5FhH41KERhc48ooxGYTln2P8YlPPhHnsEdRrrFuc2CBFQZJ4XvcT3hEd328GxILbSKHlpUpaRdrYqwCf57lJGBRksYLdyYCNxrMf83RjwY0Z2BkCQG1Gr4sOJukwqD4rDqjTBe56nYDSq8bS3qIuCIN7dlhb13G5cY/NzJCQBvzFkj2cazTOHb0pqhJHaoTxaLr+Wbi6R9xr2nxlU='} 2023-06-23T01:21:29.155 DEBUG:teuthology.nuke:shortname: smithi157 2023-06-23T01:21:29.155 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-23T01:21:29.182 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311926', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-23 00:45:09.819354', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMFEBtPcrY8XOJPHcTEL+TCv+28K01iw+hW5n3tKkPWTQ33e/kONGkSJPKr06gZsfmpiRvqSGEQASzbmBk5Nztg='} 2023-06-23T01:21:29.193 INFO:teuthology.orchestra.console:Power off smithi120 2023-06-23T01:21:29.195 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-23T01:21:29.217 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T01:21:29.248 INFO:teuthology.orchestra.console:Power off smithi134 2023-06-23T01:21:29.249 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-23T01:21:29.282 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T01:21:29.305 INFO:teuthology.orchestra.console:Power off smithi157 2023-06-23T01:21:29.306 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-23T01:21:29.332 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T01:21:33.269 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T01:21:33.320 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T01:21:33.360 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T01:21:37.384 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T01:21:37.436 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T01:21:37.476 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T01:21:37.652 INFO:teuthology.orchestra.console:Power off for smithi134 completed 2023-06-23T01:21:37.692 INFO:teuthology.orchestra.console:Power off for smithi157 completed 2023-06-23T01:21:41.500 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T01:21:41.714 INFO:teuthology.orchestra.console:Power off for smithi120 completed 2023-06-23T01:21:41.821 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 1c622d92b6f18ce44a7c33802686ff61c3c273ab description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/no standby-replay subvolume/{with-namespace-isolated} tasks/{0-check-counter workunit/suites/fsstress} wsync/{no}} duration: 1609.7314774990082 failure_reason: 'Command failed on smithi120 with status 22: ''sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5''' flavor: default owner: scheduled_yuriw@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=c84e5b4752344dc0b4da7818d9855762 status: fail success: false 2023-06-23T01:21:41.821 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-23T01:21:41.913 INFO:teuthology.run:FAIL